Turris OS is released as a set of packages, updater rules and medkit of distinct version. Such set is marked as some exact Turris OS version. To facilitate automatic updates with versioned distribution updater (tool on router to facilitate updates) follows so-called branches. This makes Turris OS effectively rollings distribution as well, although you have to understand that branch is in reality some floating link to full build of Turris OS (so not a real rolling distribution with incremental packages updates).
User can choose to follow various binary feeds. There are some pure branches, such as HBS and HBT and also branches with corresponding source branch in this git repository. In reality, binaries in HBS and HBT are built in HBK and from there only copied.
Binary branches are signed with two different signing keys. There is release key and test key. The reason is that the release key is stored more securely and requires more than one developer to receive it while the test key is available for all build jobs on build server.
Users can switch between these branches with switch-branch script on the router.
List of all primary binary branches continues. Branches are sorted in order of stability. Changes are flowing from bottom branch to top ones.
.----. @ @ / .-"-.`. \v/ | | '\ \ \_/ ) ,-\ `-.' /.' / '---`----'----'
This is stable release branch. It contains the most tested and stable version. It is end-users branch. This is default branch routers follow.
Binary packages in this branch are signed with release key.
Releases in this branch are tagged in this repository with appropriate version tag
(such as v5.1.2).
Binaries in this branch are built in HBK branch and copied to this branch from HBT.
__
.,-;-;-,. /'_\
_/_/_/_|_\_\) /
'-<_><_><_><_>=/\
jgs `/_/====/_/-'\_\
"" "" ""
This is branch used for end-users testing of next release. This branch contains basically release candidate versions.
Binary packages in this branch are signed with release key.
Releases in this branch are tagged in this repository with appropriate version tag
with -rcX suffix where X is sequence number. Content of this branch can be
potentially same as HBS if there is no other release candidate.
Binaries in this branch are built and copied from HBK branch.
("`-''-/").___..--''"`-._
`6_ 6 ) `-. ( ).`-.__.`)
(_Y_.)' ._ ) `._ `. ``-..-'
_..`--'_..-_/ /--'_.'
((((.-'' ((((.' (((.-'
This branch contains accumulated changes for next release. It is used to build and continuously test binaries that are later moved to HBT and subsequently to HBS.
Binary packages in this branch are signed with test key.
This branch corresponds to git branch hbk.
Feeds in this branch are configured to OpenWrt stable and Turris OS packages master branch. This means that this branch in general continually receives only hotfixes. The only case when it jumps and receives new features is when git branch hbl is merged in to it.
,%%%%%%%%,
,%%/\%%%%/\%%
,%%%\c "" J/%%%
%. %%%%/ o o \%%%
`%%. %%%% _ |%%%
`%% `%%%%(__Y__)%%'
// ;%%%%`\-/%%%'
(( / `%%%%%%%'
\\ .' |
\\ / \ | |
\\/ ) | |
jgs \ /_ | |__
(___________)))))))
This branch contains the latest development of Turris OS features that are based on stable OpenWrt. It is used to build and test the latest Turris features before they are merged to git branch hbk.
Binary packages in this branch are signed with test key.
Feeds in this branch are configured to OpenWrt stable branch and Turris OS packages development branch. This means that branch in general continually receives hotfixes from OpenWrt and the latest features and bugfixes from Turris OS. New features from OpenWrt are delivered to this branch by merging git branch hbd in to it.
\||/
| @___oo
/\ /\ / (__,,,,|
) /^\) ^\/ _)
) /^\/ _)
) _ / / _)
/\ )/\/ || | )_)
< > |(,,) )__)
|| / \)___)\
| \____( )___) )___
\______(_______;;; __;;;
This is the most unstable branch with all the latest development. It is used to test the latest OpenWrt development changes together with the latest Turris OS improvements before they are merged to git branch hbl.
Binary packages in this branch are signed with test key.
Feeds in this branch are configured to Turris OS packages development branch and OpenWrt development branch. OpenWrt branch in this case is either master or next not yet released stable branch. This branch is merged to hbl when OpenWrt releases new stable version.
This section describes the development flow for this repository. It is closely related to the previous section, but unless you are developer looking to contribute or maintainer of this repository then it is probably not of interest to you.
- hbk
-
is default branch with version of Turris OS that is considered stable. This branch contains stable version with only bugfixes on top of it. It is based on OpenWrt stable branch (
openwrt-XX.XX). - hbl
-
is branch used for development and contains possibly new features and larger changes. It is still based on OpenWrt stable branch, same as hbk.
- hbd
-
is branch used for development of next major version. It contains not only large changes from Turris but also automatically pulls changes from OpenWrt development branch. The exact OpenWrt branch changes between master and future OpenWrt stable.
Following flow describes how branches are merged and how repository works in general. It also describes feeds configured in that specific branch. We specify branch in OpenWrt repository (abbreviated as owrt) and branch in Turris OS packages (abbreviated as tos). There are other repositories (feeds) but they should be configured to be consistent with OpenWrt feeds configuration.
The master branch of this repository is called hbk and all branches are forked from it. Let’s follow development of Turris OS 5.0 that is based on Turris OS 4.0 and OpenWrt 19.07. Turris OS is based on OpenWrt 18.06. Initially this version is being developer in hbd branch and meanwhile hbl contains next minor release and hbk next fixup release for Turris OS 4.0
hbd hbl hbk
_______________________________________________/|
|--------------------- |--------------------- |---------------------
| owrt: openwrt-19.07| | owrt: openwrt-18.06| | owrt: openwrt-18.06|
| tos: develop | | tos: develop | | tos: stable |
|--------------------- |--------------------- |---------------------
| | |
When problem is discovered in hbd, such as that patches are broken, then fix is prepared in branch forked off of it.
fix/foo | | |
____/| | |
| | | |
| | | |
Later such branch is merged back to hbd when fix is finished and tested.
| | | |
|___ | | |
\| | |
| | |
Once OpenWrt releases new stable (marks in example case OpenWrt 19.07 as stable and 18.06 as old stable) then content of hbd can be merged to hbl. Note that after that we can’t release next minor release for previous major release of Turris OS.
|______________________ | |
|--------------- \|--------------------- |
| owrt: master | | owrt: openwrt-19.07| |
| tos: develop | | tos: develop | |
|--------------- |--------------------- |
| | |
When problem is discovered then new branch is forked off of hbl to fix it.
| bugfix/foo | |
| _______/| |
| | | |
| | | |
That is later merged to both hbl and hbd when bugfix is finished and tested.
| | | |
| ______________|______ | |
|/ \| |
| | |
To do final release of Turris OS 5.0.0 we have to first merge it to hbk and build it there. Note that this is last step before releasing first release candidate as that removes possibility to release fixup for previous major version.
| |______________________ |
| |--------------------- \|---------------------
| | owrt: openwrt-19.07| | owrt: openwrt-19.07|
| | tos: develop | | tos: stable |
| |--------------------- |---------------------
| | |\<-v5.0.0
| | |
Any problem discovered in hbk, hbt or hbs is fixed in branch forked off of latest commit in hbk.
| | hotfix/foo |
| | _______/|
| | | |
| | | |
Such fix has to be merged to hbk as well as to hbl and hbd.
| | | |
| ______________________|_______________|______ |
|/ |/ \|
| | |\<- v5.0.1
Now when new feature should be added then new branch is forked off of hbl.
| feature/foo | |
| ______/| |
| | | |
| | | |
This feature is going to be part of Turris OS 5.1.0 and when it is finished and tested its branch is merged to both hbl as well as hbd.
| | | |
| _______________|_____ | |
|/ \| |
| | |
To release Turris OS 5.1.0 we merge again hbl to hbk.
| |______________________ |
| | \|
| | |\<-v5.1.0
| | |
Now we continue with flow to release fixups as well as subsequent minor and major versions. All changes this way sooner or later end up in hbk branch and that way in binary branches and at users.
For quick orientation of maintainers in repository stable naming convention is required. Depending on what you are planning to do you can create new branch with name prefixed with one of following:
- hotfix/
-
This is fix for problem affecting hbk or binary branches hbt and hbs. It has to be based on the latest commit in hbk branch and merged to all branches (hbk, hbl, hbd).
- bugfix/
-
This is fix for problem affecting hbl that is not present neither in hbk branch or in any subsequent binary branches. It has to be based on the latest commit in hbl branch. It is merged to hbl as well as to hbd.
- fix/
-
This is fix for problem affecting hbd that is not present in neither hbl nor hbk or subsequent binary branches. It has to be based on the latest commit in hbd branch and is merged only to hbd.
- feature/
-
New featured that is supposed to be part of next minor release so that means that it has to be based on hbl branch. This can be any change that modifies packages/lists/medkit or build process itself. It is merged to both hbl as well as to hbd.
- majorfeature/
-
New feature that is supposed to be part of only next major release. This is discouraged as you probably want to release it rather in some subsequent minor update but there can be reason why that can’t be done and in that case this can be used. It has to be based on the latest commit in hbd branch and merged back to hbd.
- refactor/
-
This is same as feature/ but it should not change packages/lists/medkit or build process. This can be code cleanup, patches cleanup or any other refactor.
- majorrefactor/
-
This is combination of refactor/ and majorfeature/. Use when what you are refactoring is available only in hbd branch.
- hack/
-
This is feature that is to be reverted in the future.
- majorhack/
-
This is combination of hack/ and majorfeature/. Use this if hack should be merged only to hbd.
After prefix you should add short name for what you are about to do. Acceptable is for example name of package or feature you are about to implement. Please do not use issue numbers of nothing saying generic words (such as: hotfix/problem).
Developer is anyone who wants to contribute to this repository. Developers are not allowed to merge to primary branches (hbk, hbl and hbd) and have to ask maintainers to do so (submit pull or merge request or patch).
You have to base your work on latest commit in hbk. For example:
git fetch
git checkout -b hotfix/foo origin/hbkAlso note that you might and will be asked by maintainer to possibly rebase your changes on latest hbk commit.
git fetch
git checkout hotfix/foo
git rebase origin/hbkYou have to base you work on latest commit in hbl branch. For example:
git fetch
git checkout -b bugfix/foo origin/hblSame as in case of hotfixes you might be asked by maintainer to rebase your work on latest commit in develop branch.
git fetch
git checkout bugfix/foo
git rebase origin/developThere are well informed maintainers who are well educated with git-craft and with the flow of this repository that they are allowed to manage hbk, hbl and hbd branches. For those not so lucky and new in this craft there is following list of operations commonly performed by maintainer.
Hotfixes should always be merged to branches hbk, hbl and hbd.
git checkout hbk
git merge --ff-only --gpg-sign hotfix/foo
git checkout hbl
git merge --no-ff --gpg-sign -m "Merge branch 'hotfix/foo' into hbl" hbk
git checkout hbd
git merge --no-ff --gpg-sign -m "Merge branch 'hotfix/foo' into hbd" hbl
git push origin hbk hbl hbd
git branch -d hotfix/foo && git push origin :hotfix/foo|
Important
|
Push target first before you remove source branch. Otherwise GitLab merge request would be closed and not merged. |
Hotfix merge to hbk should always be clean. That means that there should be no merge conflicts (ensured by requiring fast forward only). This is to ensure that stable release won’t be broken by merge. When that can’t be done rebase to latest changes has to be performed. The maintainer can either ask author or do it by himself.
There is possibility that hotfix is not required in hbl or hbd branch because
it can be fixed by some other means or was already fixed by some previous feature.
In such case it should be merged anyway to ease future merge of hbl and hbd
back to hbk. For doing merge without merging changes you can use git merge
strategy ours.
git checkout hbd
git merge --no-ff --strategy=ours --gpg-sign hbkBranches with new features, bugfixes, refactoring or hacks are merged to hbl branch. This merge should be without conflict as well to prevent bugs created by merge skipping testing. On merge conflict it should be evaluated, rebased on latest commit in hbl (to resolve conflict) and test again before merge.
git checkout hbl
git merge --ff-only --gpg-sign feature/foo
git checkout hbd
git merge --no-ff --gpg-sign -m "Merge branch 'feature/foo' into hbd" hbl
git push origin hbl hbd
git branch -d feature/foo && git push origin :feature/foo|
Important
|
Push target first before you remove source branch. Otherwise Gitlab merge request would be closed and not merged. |
Branches with new major features, fixes, major refactoring or major hacks are merged to hbd branch. This merge should be without conflict as well to prevent bugs created by merge skipping testing. On merge conflict it should be evaluated, rebased on latest commit in hbd (to resolve conflict) and test again before merge.
git checkout hbd
git merge --ff-only --gpg-sign majorfeature/foo
git push origin hbd
git branch -d majorfeature/foo && git push origin :majorfeature/foo|
Important
|
Push target first before you remove source branch. Otherwise Gitlab merge request would be closed and not merged. |
Hacks are intended to be present only temporally and so it is expected that in the future we want them reverted and that way removed. This means that hacks are merged as an exception with merge commit instead of doing fast-forward merge.
git checkout hbl
git merge --no-ff --gpg-sign feature/foo
...Later hack, thanks to merge commit, can be just reverted by locating appropriate merge commit hash and reverting it. Note that this is considered as refactor and new appropriate merge request should be created and review should be performed.
git checkout -b refactor/foo origin/hbl
git revert -m 1 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxAll stated here also apply on major hack with exception that target branch is hbd instead of hbl.
Turris OS minor releases are prepared in hbl branch but all changes introduced have to land to hbk before they are propagated for testing to HBT and later for final release to HBS. Thanks to previous steps this should be merge without conflicts.
git checkout hbk
git merge --no-ff --no-commit hblDo not create commit immediately as it is required to modify few files:
-
Change
PUBLISH_BRANCHvariable value tohbsin filedefaults.sh. -
Change branch from
developtomasterforturrispackagesfeed infeeds.conffile. -
Change branch from
developtomasterforupdater-listscomment infeeds.conffile.
Now add changes to the git staging area and create a commit.
git add defaults.sh feeds.conf
git commit --no-edit --gpg-signThe last step before the push to the server has to be propagation of changes to hbl and hbd branches. Failing to do so would reset defaults for those branches with next merge.
git checkout hbl
git merge --no-ff -m "Merge Turris OS 4.2.0 into hbl" hbk
git checkout hbd
git merge --no-ff -m "Merge Turris OS 4.2.0 into hbd" hbl
git push origin hbk hbl hbdLastly all changes are pushed to server.
|
Important
|
At the same time it is required to merge branch develop to master in the turris-os-packages and updater-lists repositories! |
Turris OS major releases are based on major versions of OpenWrt. HBD branch collects fixes and changes based on latest OpenWrt development and thus when these changes became stable thanks to OpenWrt release they have to be moved/merged to hbl and hbk. At some point it is decided that there is going to be no new minor version of Turris OS based on at that time old stable version of OpenWrt and major release is going to be performed. This can take considerable amount of time during which it is desirable to still be able to develop and release fixes. This means that first step has to be move of all changes from hbd just to hbl. This is analogous to move of hbl to hbk:
git checkout hbl
git merge --no-ff --no-commit hbdCommit is not created immediately as changes to defaults have to be reverted:
-
Change
PUBLISH_BRANCHvariable value tohblin filedefaults.sh.
git add defaults.sh
git commit --no-edit --gpg-signThe last step before the push to the server has to be propagation of changes back to hbd. Failing to do so would reset defaults for those branches with next merge.
git checkout hbd
git merge --no-ff -m "Merge Turris OS 4.0 into hbd" hbl
git push origin hbl hbdLastly all changes are pushed to server.
Subsequent merge to hbk is performed the same way as if minor release of Turris OS is being released.
The release itself is performed outside of this repository. In effect files in HBK branch are taken, all signatures are updated and all is deployed to HBT.
Before release the verification should be performed using dedicated script:
./helpers/new_release.sh verifyThis checks if all boards are in sync. It is acceptable to do release of release candidate even when not all boards are in sync but such build should not be pushed to HBS.
The final release itself is just move of files from HBT to HBS.
The final releases have to be tagged in this repository as well as in turris-os-packages repository. There is helper script implemented to correctly create tag in this repository. Tag is not automatically pushed. It should be reviewed and pushed manually.
./helpers/new_release.sh release
git push origin vX.Y.ZThis is collection of various tips and primarily configuration options you can use to simplify commands described in this flow.
- Use project specific git configuration
-
It is highly advised to use project specific git config. You can apply it by running following command:
git config --local include.path ../.gitconfig- Sign commits and tags with GPG without using
--gpg-signand-s -
You can configure global or local git option
commit.gpgSignandtag.gpgSign.
git config --local commit.gpgSign true
git config --local tag.gpgSign true- Sign commits and tags with specific PGP key
-
If you have more than one PGP key (for example different for personal and work identity), you can specify exactly which should be always used in git configuration option
user.signingKey. Value of that option is fingerprint of your PGP key.
git config --local user.signingKey "XXXXXXXXXXXXXXXX"