Decide policy for Development and Stable versions

[130s]

Moved from start-jsk/openhrp3#43

By the way, how do we cope with start-jsk/hrpsys#75, they want to start using trunk version of hrpsys.

One idea is to create groovy branch on rtmros_common and stop developing/releasing and using hrpsys 315.1.x and hydro branch is continue developing on trunk (315.2.x), but this means hironx users must use groovy, so it may not acceptable.

Another idea is to create stable(315.1.x) branch on rtmros_common and only stable branch is released and deb build. This means main/core users never uses deb package. Also I afraid I mostly working on developing release so it just make you much more busy.

Another approach is to release rtmros_common and rtmros_comon_devel ?

Ideal approach is to update QNX software to 315.2.x but it may most difficult among above idea.

[garaemon]

If we have hrpsys-trunk (hrpsys-head?) branch on hrpsys and rtmros_common (and potentially on rtmros_gazebo),
we don't need to wait hrpsys relase in order to use new features.

However, managing the code might be more difficult.

For example, we developer will send PRs to hrpsys-trunk branch and the maintainer needs to merge hrpsys-trunk with master when a new hrpsys is released.

[k-okada]

Currently we have following repositories

• hrpsys-base (fnakeniho/hrpsys-base)
• hrpsys (start-jsk/hrpsys)
• rtmros_common (start-jsk/rtmros_common) (rtmros_gazebo will be same)

and we presume

• experts : YOU. they're ok to compile everything with source code.
• beginners: I want them to use dep ( I don't want to see new comers spent one week just to compile, or spend one lecture just to compile), they're ok to update dep often.
• customers: HIRO users, then use dep and basically they do not want to update, because of the riks of break system. Also compile hrpsys on QNX is a problem and we still do not have good delivery system of QNX binary to their computer. (ex. PR list for hrpsys trunk #75 (comment))

hrpsys-base: Currently hrpsys-base have 315.1.x, 315 correspond to OpenHRP3.1.5 and 1 correspond to hrpsys's IDL revision, so if we changed IDL, it going to 315.2.0. and current master have different IDL than 315.1.x so next release will be 315.2.0.
Last few month, since we release 315.1.x, I pushed to useful patches to both 315.1.x and trunk https://code.google.com/p/hrpsys-base/source/detail?r=1032.
updating micro revision (315.2.0 to 315.2.1) will happens whenever we have new features or codes. This will occur very often.

hrpsys: this is just ros package for hrpsys-base, If you're expert, you just change
https://github.com/start-jsk/hrpsys/blob/master/Makefile.hrpsys-base#L7 to use trunk version of the hrpsys-base. The question is which version should we release as deb? 315.1.x ? or trunk? for beginners and customers.

rtmros_common: this depends on hrpsys-base versions and if we have rtmros_common for 315.1.x, this package could not talk to hrspys-base 315.2.x(trunk). One idea is to create devel branch that contains code designed for 315.2.x and continue release(debianize) on master branch that is for 315.1.x.

Another approach is to create ubuntu repository mirror for customers. That sync from original ubuntu repository whenever they confirmed to work on real robot. Cons. of this approach requried to explain: "ok you have hiro/torka- repository that have different setup than original ros/ubutnu repository so the code used for original repository may not work on your computer".

[130s]

I feel like doing more research on this since there are plenty of mature software projects we can learn from, and I'm not experienced in this regard either. But some people might be already imperative, so here's my comment from "industry" side.

Another idea is to create stable(315.1.x) branch on rtmros_common and only stable branch is released and deb build. This means main/core users never uses deb package. Also I afraid I mostly working on developing release so it just make you much more busy.

Another approach is to release rtmros_common and rtmros_comon_devel ?

Both sound reasonable and interchangeable with each other.

I think the key question is "how often do we want to release?" And, so far I feel that the requirements from research users (JSK) and industry users are different; researchers appreciate more frequent release while industry users often require stable version. That said, separating .deb into normal and _dev might make the best sense.

Below are my supporting thoughts:

• Separating devel (or trunk. I'll keep saying as devel on this post for brevity) and release branches should be fairly common in mature projects (e.g. In boost, release is only made from release branch. See "Development and Release Practices").
• Merging from devel into release is what the maintainers/release managers are responsible for.
• Core developers don't use .deb of the package that they are working on anyway.
• Split branches won't increase the amount of release work, since devel won't be touched by release person.
• Just want to spread the source tree (e.g. hrpsys) so that everybody shares consensus.

hrpsys
|-branch
  |- trunk or devel (and %DISTRO%-devel is common in ROS)
  |- release or master
|-tag
  |- hrpsys315.1.9
  |- hrpsys315.1.10
  :

[k-okada]

• trunk or devel (and %DISTRO%-devel is common in ROS)
• release

You may have heard same thing agin and agin. I think I could not support / maintain two different code set. If there are someone who can fix a bug, answers to the issues at least 30% of the project, I'l happy to say yes. But at this moment I'm really afraid that splitting branches breaks both the rtm-ros project and my life.
https://github.com/start-jsk/rtmros_common/graphs/contributors

This is my personal feeling and if there are people who is able to do that, that good news for me. But remember, if you read this issue right now, you may continue working more that 16 hours, do you think you have another few hours to do new thing??

This is comment, not only this issue, but in general on research and code development

• once you created branch, it very hard to merge, that is the entropy, the nature. Foe example, see [1], that started from same setup as we have: hrpsys, ROS, opencv, Kinect, but what he created is completely different world.
• even if you think something is easy not not a hard job, it is very hard to find people who is able to do that as well as who has time to do that. For example [2][3], seems very easy job, may be a few lines, but leave unsolved for months.
• I think only criteria is to ask our selves that "is that saves your time/line in three month later?", if this saves current time, it may just postpone the current workload to the feature. if you think it will save your time 6 month later, it will never happens.

[1] : https://github.com/hyaguchijsk/jsk_recognition/tree/feature/clothbag_rim_detector/clothbag_rim_detector
[2] : start-jsk/rtmros_hironx#36
[3] : start-jsk/rtmros_common#315

[k-okada]

so here's my comment from "industry" side.

This is just another point of view and just discussion

In start-jsk/rtmros_hironx#76, I just forget to install the robot tools, however if everyone can use these tools easily, then there is a risk of no one will ask you about supporting the robot. The reason users ask for supporting service may:

1. they can not do that
2. they have no time to do that
   and closed approach may work for people in group (1). Even you do not close the software, if the software is too complex and messy, then "they can not do that". This means, if the software changed so rapidly and they can not follow what happening but it seems getting better and better, then they will ask you for support.

So from "industrial provider" side, Is there any chance that faster release become good???

By the way, how many people want support due to (1) and how many for (2).

[k-okada]

Merging from devel into release is what the maintainers/release managers are responsible for.

I think maintainer will use cherry-pick to do this type of jobs, also when we have useful or critical patch to one devel-$ROS_DISTO branch, then we want to commit that one to another branch.

In that case, if the PR contains many commits, is there way to chery pick mutliple commits as
git cherry-pick XXXXX:YYYYYY ? or we're expecting that contributors kindly create cleaner PR as
https://github.com/ros-planning/moveit_ros/pull/431/commits ?

Also when we maintain software, will maintainer expect that contributor will write test code for their commits? Or is this maintainer's job to write test code?
At least at jsk-ros-pkg, it seems maintainer's job....
-> jsk-ros-pkg/jsk_common#305 , I said to write test code and finally @garaemon wrote
-> jsk-ros-pkg/jsk_pr2eus#10, @garaemon said to write test code and finally I wrote
May be we're killing each other....

Is there any best practice to ask contributor to write test code without discouraging them?

[garaemon]

another solution is to create hrpsys-trunk locally on the machines related with the real robot.
Because I think the problem is switching/merging PRs when we use hrpsys-trunk.

[garaemon]

about test code;
One solution is Test-Driven-Development. However I don't agree with the strategy at this time.
Because we cannot see API and behavior before implement it and we are writing state-of-art software (I believe)...

[garaemon]

Today we updated the source codes inside of the real robots and we found that the work was painful.

We said 'Did you merge that PR?" a lot of times.

So I suggest that creating branch for hrpsys trunk on rtmros_common, rtmros_gazebo and rtmros_tutorials.

we don't need hrpsys-trunk branch on start-jsk/hrpsys, because the diff is just a one line.

I prefer hrpsys-trunk to devel as the name of the branch.
Because 'devel' might be misleading name in this case.
hrpsys-head, hrpsys-latest are welcome for me.

So we will have hrpsys-trunk, and we need few policies about sending PRs.

1. merge master to hrpsys-trunk everyday. because hrpsys-trunk should include the latest patch on master.
2. merge hrpsys-trunk to master every release of hrpsys.
3. sending PR to hrpsys-trunk if it uses new features on hrpsys trunk.
4. sending PR to master if it does not uses trunk features.

about 1 and 2, we can use cherry-pick but I'm not so good at git magical commands. another solution is sending PR to sync these two branches.
Update users remote branch to the latest and create a PR from user/master to origin/hrpsys-trunk.

[garaemon]

If you guys have no opinion against the hrpsys-trunk branch, I will create that branch tonight.

[k-okada]

merge master to hrpsys-trunk everyday. because hrpsys-trunk should include the latest patch on master.
merge hrpsys-trunk to master every release of hrpsys.
sending PR to hrpsys-trunk if it uses new features on hrpsys trunk.
sending PR to master if it does not uses trunk features.

this seems good, just to clear, hrpsys-trunk is the branch in rtmros_common and hrpsys trunk is the maser in fkanehiro/hrpsys-base.

merge master to hrpsys-trunk everyday. because hrpsys-trunk should include the latest patch on master.

this must be done in automatically, also we'd better to check if the PR is really requires hrpsys trunk, otherwise the PR should be rejected.

merge hrpsys-trunk to master every release of hrpsys.

do you mean `every release of hrpsys (fkanehiro/hrpsys-base)

if so, if we release next 315.1.x, the we can merge `hrpsys-trunk` to `hrpsys`, and if next release is the `315.2.x` then `hrpsys-trunk` no longer needed.

[k-okada]

So I suggest that creating branch for hrpsys trunk on rtmros_common, rtmros_gazebo and rtmros_tutorials.

Do we really need to create hrpsys-trunk branch for rtmros_gazebo and rtmros_tutorials?

[k-okada]

by the way, do we really create hrpsys-trunk at this moment?
from #75
start-jsk/rtmros_common#396 -> this is only euslisp, so it is ok to merge (currently we can assume people who use euslisp will compile everything from source)
start-jsk/rtmros_common#397 -> If the change is just change RemoveForceSensorLinkOffset -> AbsoluteForceSensor, then it will be ok we backed the name in the hrpsys-base/master
start-jsk/rtmros_common#424 combination of 396 and 397
start-jsk/rtmros_tutorials#37 as long as hrpsys_ros_bridge.launch did not includes robot_pose_ekf.launch, it seems ok
start-jsk/rtmros_common#430 I'm not sure what is this PR is, but if https://github.com/start-jsk/rtmros_common/pull/430/files#diff-ac73db3c66ec2afe6e6410a14af1ebc0R533 is the change, the it seems ok.

I think we'd better to tuckle on start-jsk/rtmros_common#416, so that we can check if merged version did not break downward packages.

[garaemon]

I think we should have hrpsys-trunk branch.
Because this kind of discussion is already too difficult for the users.

just saying 'use hrpsys-trunk' is simple enough rather than 'merge these 4
PRs'.

2014年4月19日土曜日、Kei [email protected]さんは書きました:

by the way, do we really create hrpsys-trunk at this moment?
from #75 #75
start-jsk/rtmros_common#396https://github.com/start-jsk/rtmros_common/pull/396-> this is only euslisp, so it is ok to merge (currently we can assume
people who use euslisp will compile everything from source)
start-jsk/rtmros_common#397https://github.com/start-jsk/rtmros_common/pull/397-> If the change is just change RemoveForceSensorLinkOffset ->
AbsoluteForceSensor, then it will be ok
start-jsk/rtmros_common#424https://github.com/start-jsk/rtmros_common/pull/424combination of 396 and 397
start-jsk/rtmros_tutorials#37https://github.com/start-jsk/rtmros_tutorials/pull/37as long as hrpsys_ros_bridge.launch did not includes robot_pose_ekf.launch,
it seems ok
start-jsk/rtmros_common#430https://github.com/start-jsk/rtmros_common/pull/430I'm not sure what is this PR is, but if
https://github.com/start-jsk/rtmros_common/pull/430/files#diff-ac73db3c66ec2afe6e6410a14af1ebc0R533is the change, the it seems ok.

I think we'd better to tuckle on start-jsk/rtmros_common#416start-jsk/rtmros_common#416,
so that we can check if merged version did not break downward packages.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/76#issuecomment-40861891
.

from iPhone

[k-okada]

I did not say merge this one to your local branch, but I said that we could merge this one into master branch. Some of PR does not depends on hrpsys-trunk.

Plese see #75 and ask @snozawa if he can merge the code into 315.1.x without braking API

[130s]

Yesterday I talked to @k-okada and understood the situation better. I just spread out how I understood it and if this is correct I would +1 for creating hrpsys-trunk branch.

• Master branch is a subset of trunk (no commits in master should break trunk). When a new commit is made to master, a maintainer can merge it into trunk.
• Hrpsys and rtmros_common packages need branched.
• Trunk branch can merge master, while master needs to carefully cherry pick from trunk.

[k-okada]

I think about this issue over weekend and have another idea:
currently we have

(1)

(hrpsys) ( rtmros_common )
master
315.1.x -> deb/qnx --- master -> deb

and what we discussed here is
(2)

(hrpsys) ( rtmros_common )
master --- hrpsys-trank
315.1.x -> deb/qnx --- master -> deb

I have still worried about few things on this approach
A) merging / cherry picking between these branches might be hard work, so old version going to be freeze
B) A) means if we have very nice improvement on new branch people using old branch will not benefit from that
C) development user can not use deb, that is not good. because expert developer works so hard and changed so many code, so that people working with him tend to become just compile ( and fix compile error ) all the time, they should use deb package.
(2)'

(hrpsys) ( rtmros_common )
master --- hrpsys-trank | development
315.1.x -> deb/qnx --- master -> deb | end-of-life (or you can say stable)

But when we close look at hrpsys, we can classify RTC to stable one and experimental one. For example SequencePlayer is stable one but AutoBalancer is experimental at this time. So far, I thought if you changed any of IDL file, then it will break hrpsys / rtmros_common connection, but it might be if you changed any of Stable RTC's IDL, then it will break, but changing experimental RTC's IDL is ok for stable users. If so, we can create new version of hrpsys, say 315.2.x whose IDL is different from 315.1.x, but if we look only Stable RTC's IDL it is as same as 315.1.x. and this version will be compatible with old version of hrpsys as follows.

(3)

(hrpsys) ( rtmros_common )
315.1.x -> qnx. --\
315.2.x -> deb --- master (compiled from 315.2.x)-> deb

I've create CI job to check this situation at https://travis-ci.org/k-okada/hrpsys-base/builds/23430636

But another question is when we'll release new version 315.3.x, which have different stable RTC's IDL? I think it should be synchronize to ROS/Ubuntu release, so it means now.

So it going to be like this.

(4)

(hrpsys) ( rtmros_common )
master I --- hrpsys-trank I | development
315.1.x -> G/H - deb/qnx --- master -> H - deb | end of line

Difference between (2) and (4) is, (4) can release deb file so that we can solve (C) problem. (A) and (B) staill remains, but two people , one working on H/old and others working on H/new, then they will think like why can I use others's code? but if they working on different ubuntu release, may be they will accept.

If we think this direction , we 'll use newer hrpsys version strategy that
314.1.x is stable, so no IDL has changed(no stable IDL? or not all IDL?), and 315.2.x is experimental, so all IDL (including stable version) can be changed during this version.
Then we release stable version 315.3.x (off course I'll skip hrpsys version to even number become stable as Ubuntu),

and release time line is become something like this.

14.04 15.04 16.04 17.04
... 315.2. 315.3
I -----------------
................ 315.4
............. J ------- 315.4 315.5
.............................. K ------------
..................................... 315.6
..................................... L ------

So the rules is

• first 1 year of the ROS release is unstable for rtm-ros, so you can change / break anything
• but the lest of the (first) year of ROS release, that's is going to be stable rtm-ros

In order to do that

• developers ( for example @garaemon, @snozawa, @YoheiKakiuchi), you'll use 14.04/Indigo for development ( I already get negative response from @garaemon, ...), not to break stable version.

I'm afraid this direction is too stress for industrial players, because all of active developer's efforts is done in experimental revision and they won't care about backward compatibility.

• Industrial players (for example @130s), this means newer development is mainly working on developed version, so issue like hironx_moveit_config does not depend on collada_tools rtmros_hironx#65, Color me! Model rtmros_hironx#48, Turning off relay switch automatically turns off servo rtmros_hironx#38, create GUI that restart QNX / rtcd rtmros_hironx#37 will leave unsolved. Or even it is solved, you have to find by your self, for example one of the problem was solved recently but can you find from https://github.com/start-jsk/rtmros_common/commits/master ? (.. the answer is, use rqt on hrpsys_dashboard rtmros_common#434 and there are more improvement at garaemon/rtmros_common@9764d48) I'm afraid that it is not a easy job.

[garaemon]

it semms too tricky to separate deb according to the ubuntu distribution (What a progress I say "too tricky"!).

I'd like to keep things as simple as possible.

Just idea, if we have hrpsys-trunk and master, is it possible to create PR automatically using
cherry-pick?

auto generate following PRs.

• PRs master -> hrpsys in order take into account the change of the latest master.
• PRs hrpsys->master in order to take into account the fix for hrpsys-trunk (It occurs when release new hrpsys ).

[130s]

As dealing with industry users, I think I'm fine as long as:

• stability (module and API) of .deb is checked.
• there's always a way to get new features (preferably via .deb but for urgent cases source build is acceptable).

Industrial players (for example @130s), this means newer development is mainly working on developed version, so issue like start-jsk/rtmros_hironx#65, start-jsk/rtmros_hironx#48, start-jsk/rtmros_hironx#38, start-jsk/rtmros_hironx#37 will leave unsolved. Or even it is solved, you have to find by your self, for example one of the problem was solved recently but can you find from https://github.com/start-jsk/rtmros_common/commits/master ? (.. the answer is, start-jsk/rtmros_common#434 and there are more improvement at garaemon/rtmros_common@9764d48) I'm afraid that it is not a easy job.

I'd rather like to give this a shot how we can handle this proposed situation. I'm afraid core developers might want to figure out / end this discussion asap and get back to their work?

---

From here are just comments.

C) development user can not use deb, that is not good. because expert developer works so hard and changed so many code, so that people working with him tend to become just compile ( and fix compile error ) all the time, they should use deb package.

I think core developers use source build anyway aren't they? I thought users are:

• Core developers use source build (currently a handful: @k-okada, @garaemon, @snozawa, @YoheiKakiuchi, @mmurooka, @eisoku9618 ?)
• Core research users (eg. novice users at JSK?) choose source build/.deb depending on the situation (this could be more than a dozen)
• Industry users are advised to use .deb.

Having stable API per major release (ie. ROS distro) is ideal, not just as a ROS package but in Ubuntu or I assume in any software devs, but because the user base isn't yet that huge, I think we can work it around by making complete announcement (and support for industry users) if API breaking change is a must.

[garaemon]

Core developers use source build (currently a handful: @k-okada, @garaemon, @snozawa, @YoheiKakiuchi, @mmurooka, @eisoku9618 ?)
Core research users (eg. novice users at JSK?) choose source build/.deb depending on the situation (this could be more than a dozen)

It's difficult to separate core developers and core research users because we share the same
robots and if the research users want to use new features, they are forced to use source version.