Branch Development Fallacies

I have now been on 2 major projects within Australia that have used the branch development philosophy. The idea behind this is:

  1. Project is defined by business
  2. Copy of main branch is created and made available to new Project team
  3. All development for that project is confined to the branch
  4. When new changes are committed to the main branch – maintenance fixes or other projects going live – each branch merges the changes from main into their branch and deals with any conflicts
  5. As the project nears completion, it is assigned a production deployment date alongside one or more other projects
  6. Projects assigned to the same production deployment date merge together into an integration branch which is used to create deployment artefacts for testing that release
  7. Development for each project continues in isolated branches – code is merged into integration branch when teams ready to commit
  8. Once project goes live, the integration branch then gets merged back into the Main branch and all other outstanding development branches pull the changes into their codebases

In the ideal situation as outlined above, it makes perfect sense to the business. They argue that this type of development reduces risk by:

  • Making it simple to cancel a project and not have to worry about removing any completed code from the codebase
  • The main branch will always be stable
  • Projects that miss their deadlines or are parked in order to develop higher priority work do not adversely impact the main branch and the ability to push new releases

There are 2 problems with this type of thinking. The first is that very few long running projects that have cost millions of dollars are ever going to get cancelled. More likely they will get pushed to the next release but that means that all the testing that has taken place against the integrated code base must now be re-executed, likely leading to the other projects in that integration branch also missing the deploy window. The second is that while the approach may decrease business risk in that projects can be more easily managed, it actually increases technical risk. The problem with keeping each team isolated on its own branch until the last moment are:

  1. late integration – Since we are waiting until almost the end of a project to start merging our code with the other development branches going into Production with us, it gives us much less time to effectively test the codebase and resolve and merge issues that should arise. Regression defects are much more likely to be introduced as functionality that previously worked in one branch in isolation now breaks against the merged codebase. The teams involved now need to get together and hash through code that was potentially written months ago to determine why the functionality broke, who is responsible and how to get it fixed while preserving the changes both teams made.
  2. code divergence – When branches are used for very long projects, each branch will make changes that take it further and further from other branches. Though each will still remain in sync with Main, when one of the projects goes into main, the merge pain encountered by the other projects will be significant because of the large amounts of change between the old main and the new main.
  3. stale branches – If a branch gets parked in favor of other branches and remains so for any significant length of time, not only will the code on it become stale but also the knowledge of the developers who created that code will also become stale. After a short amount of time, the work will become effectively lost as the developed code no longer has any owner and is no longer in sync with the main branch.

I’m not completely against branch development but I think it should take one of 2 flavours:

  1. Discrete Functionality Branches – Branches that only exist for a very short amount of time (1-2 weeks) which are used to implement a discrete bit of functionality that can then be pushed back into the main branch and be deployed if necessary. The only issue with this type of setup is that the business must be reconfigured so that they are able to break down large projects into smaller components that can be deployed independently. This approach lends itself well to continuous deployment since each new piece of functionality can be pushed out to production as soon as it has been effectively tested.
  2. Early Integration – Ideally, a business would identify projects they want to be deployed into Production together as soon as they are approved to begin development. In that way those teams could either all share a single branch for development or, alternatively, each have a separate branch but create the integration branch immediately and merge to the integration branch as often as possible. In this way any integration pain would be encountered immediately and not at the tail end of a project

There is no magic bullet here but I think that Project managers have to take a longer view than just considering business risk when they insist on developing functionality in isolation. There is a lot of technical risk that will likely land up effecting them that is not taken into account.