How to transition from a complex branching reality to a single-branch model?

Question

In large organisations, using the waterfall methodology typically results in very complex branching structures (aka branch spagetti).

What branching strategies can be used to transition from a complex branching reality to a single-branch model like trunk-based development?

Update:

To clarify, the question is about the migration/transition itself, not about the before and the after methodologies, which are pretty clear.

It can't really be "at EOB today we're still waterfall with gazillion of branches but tomorrow 1st thing we'll switchover to trunk-based, single-branch CI".

Answer 1

Because you mention waterfall, I understand that the numerous branches you are alluding to are feature-branches rather than maintenance-branches.

In this setup, I also assume that these branches are created according to a waterfall plan that tries to minimise conflicts. This implies that the goal of the development is to produce several distinct products. When using a single-branch development model, it is important to also work on a single product. If several products are simultaneously developed in a single-branch development model, it effectively “glues” together versions of theses products, so that we might have in version a of the repository a healthy product X and a buggy product Y, while in version b the product X has a regression and Y a bug fix, but we have no version where both X and Y are healthy. Such a situation would force us to consider X and Y as being developed in distinct repositories, which is a hint that they should be.

Therefore, the first steps should perform a repository split:

1. Arrange the repository so that it is easy to split it into several small repositories. For instance, rearrange the current repository so that each top-level directory corresponds to a repository you want to create in the future. Doing so, you can continue to use the branch-spaghetti discipline everybody is familiar with.

2. When step 1 is completed, refine the branch-spaghetti discipline by requiring that any single branch can only touch files in one top-level directory.

3. When each branch complies with step 2, perform the split. Developers can easily convert their pending changes to patch a single repository, just by removing the first-level of the path.

Now that the split has been performed, you can start working on the branch discipline itself.

4. Introduce programming techniques helping the development of short-lived branches. Branches being short-lived is a crucial aspect of all single-branch methodologies. One of their goals is to reduce the time spent on merging and debugging long-lived branches. A popular technique is the introduction of “feature-flags” where a “factory” uses a configuration flag to either produce the historic version of an object or the new, initially partially developed, version of that object.

5. By now, you have zillions of repositories with only a few branches in each, and you can turn the “we globally adopt the trunk-based development discipline” button, without seeing the original branch-spaghetti mountain collapsing on the trunk.

The actual split of repositories might be optional, but you would then have to adopt policies that cleanly delineate the allowed scope of each submitted patch (to limit the risk of conflicts when merging changes in the main branch). Reducing the overhead bound to conflicts is one of the goals of single-branch model methodologies, so I assume this is relevant in your context.

Answer 2

When migrating from something to something else, there are only two things you need to define:

1. What is your target
2. How to get there (the migration plan)

The first part is, sadly, often overlooked or way too vague. You cannot simply say that what you have is a mess and you want to organize it. What would that mean? Everybody would have a different interpretation (aka: every dev thinks that his or her way of doing things is the best).

Chances are, all the branches you have are serving or have served a purpose. Without a clearly defined target process, people will keep doing what works for them the way it suits them best (and rightly so).

For example, your target should be defined as clearly as Vincent Driessen defined his "successful Git branching model". If you look at this model, it is very precise: It says where stable code should be, and where unstable features should be developed. It also says how - and when - to branch, update and merge back. You know what each branch is for, and what to do with them. We use a variation of what was put forward by Vincent and our variation is defined in our wiki.

The important point is to have all the team understand and agree on a target. It might be worth it to remind people that you are not looking for their personal favorite branching model, but a model that all the team members can agree on and use easily.

Once you have your target, you'll be able to elaborate you migration plan. That plan can be as long or as short as you'd like. I've seen such branching model imposed overnight; at other places, it was done over 2 or 3 sprints. It doesn't matter much to me, as long as we are improving.

You can start with the "biggest" or more important branches. E.g.: "from now on, master must always be in a state to be deployed in prod and the dev branch must always compile" (or whatever your rules). Then, enforce version (release) branches. Afterwards, enforce feature branches. After that, impose a code freeze on version branch, if it make sense.

DevOps is all about communication, openness and efficiency. These concepts must be kept in mind and communicated throughout the process.

I would suggest to invite some people outside of the development team to the process meeting as observers. Ops or middle management might have a thing or two to say about your model. The developers' needs should be prioritized, but if the branching model is impossible to align with the way things are managed, you'd be better knowing now and not in a month or two.

If you have really big teams, try to include everyone nonetheless. With very big teams, you'll end up with two or three meetings anyway. So invite team leaders in the room, but have a webcast available and let everyone know about it. If anyone has a suggestion or concern, they'll be able to voice it to their team leader and if it is valid, it will be addressed on the second or third meeting.

Answer 3

It is actually very simple to convert a multi-branched hydra repository into a single branched model.

First, you want to start with the branches which have the least difference between itself and master or trunk. Examine their age and relevance. If they are still relevant, start merging them together and resolving conflicts. If they are no longer relevant, then delete them.

Continue this process until you have managed to merge all your branches, resolved all conflicts, and you have only a single branch remaining.

You can follow this simple outline to get started:

1. Create a copy of your master/trunk branch and call it temp_master
2. Find the branch with the greatest divergence from master/trunk.
3. Determine if the branch needs to be kept, archived, or deleted.
   1. If it needs to be kept, continue to step 4.
   2. If it needs to be deleted or archived, delete and archive it, and then return to step 2.
4. Repeat step 2 to find the next branch with the least divergence.
5. Merge the two branches found in step 2 and step 3, resolving all conflicts.
6. Merge your these two branches into your temp_master branch.
7. Test the code in temp_master code to see if it compiles, and builds, and run any other automated tests you have for sanity.
   1. If any tests fail, go back, find out why, and fix them, and repeat the process.
   2. If tests still fail, pick two different branches to work with.
8. Repeat steps 2 - 7 until you have only two branches, your master/trunk and temp_master.
9. Finally, merge temp_master into master/trunk and live with your new single-branch model.

Answer 4

For large organisations with typical 4 weeks sprint cycle Git-Flow is preferred approach because You get benefit of Feature branch Master production ready branch is always deployable Also, master branch is kept clean from unwanted commits by following two commit cycle (from feature to Develop and Develop branch to Master).

Moreover, branching is also determined by frequency of production releases. For frequent deployment to production its better to have a Feature branch or Centralized model. In this case overhead of managing branches is shifted to robust testing in lower environments to maintain production stability.