10

How do you avoid a branchageddon situation when working with large organisations?

We work with a number of large financial organisations whose approach is to not take updates to software, but instead only high/critical security patches and bespoke functionality. These organisations will only take patches and custom release in between major updates. Major updates can be years apart and carry high costs. This approach causes us (the software house) to have a branch of our code per major customer, which carries all the costs and inefficiencies of long term branching.

My questions to the community are:

  • Have you experienced similar update acceptance approaches from your customers?
  • What suggestions do you have to help work with this approach?
  • What suggestions do you have to help change organisations approaches to taking software updates?
Improve this question
3
  • Hey Mark, it sounds like you have an interesting dilemma. How do you manage developing these per customer updates? Do you develop them on a one-off basis for each customer, or is it something that you develop and apply to all customers?
    – Preston Martin ♦
    Commented Feb 21, 2018 at 14:32
  • Personally, I might look for other employment in this situation. This sounds like a security incident waiting to happen... I can tell you for the appliance vendor I worked for, they had a major bug that was fixed in an update that they reportedly could not go to. They wanted a custom fix. We refused to build them one and told them that they needed to go fix their business policies - we weren't going to roll a custom hotfix for a bug we had already patched just so they could avoid internal political issues.
    – James Shewey
    Commented Feb 21, 2018 at 18:31
  • We manage the custom client specific updates via multiple code branches and cross fixing security updates to all branches, and cross fixing branch code back into trunk. Often customer A will not take customer B updates in their branch, they will only take their own updates and security patches. This is driven by a desire for stability in their branch and so they only have to test updates relevant to them. They take trunk updates less frequently (i.e. months to years) when they are ready to run complete test reruns, which can take months to complete. Automated testing could be the answer!
    – Mark Wheeler
    Commented Feb 22, 2018 at 16:02

2 Answers 2

Reset to default
3

As Michael mentioned, offer a standard solution based on release versions/numbers, with a reasonably long lifespan for your industry (maybe interleaved with one or more shorter lifespan intermediate versions, if it makes sense for your typical customers).

Give your customers the option to embark on this standard release track, maybe with a decent migration deadline in place.

If they insist on a full-custom branch support strategy just charge them accordingly, to properly cover all your extra costs for offering such full-custom support - it only makes business sense. Some customers will migrate to reduce their costs (which will help you reduce the number of custom branches), some won't.

Variable support billing, growing progressively with the age of the release versions from which the custom branches originate can also be an incentive for customers to migrate to newer branches faster, helping with faster closing of the older custom branches. This can help reduce the number of custom branches per customer - if you have customers that simultaneouly run multiple versions of your software.

Make sure you don't fall into the trap of doing full branch merges from/to any of the release branches (both standard and custom), all changes to them should be either individually developed or cherry-picked merged fixes.

Because each of these branches will gradually diverge from each-other, the number of hotfixes requiring customisation/individual development will grow exponentially (plain cherry-pick merging will fail). You need to take the development cost for these into account.

With no (significant) branch merges in the picture you can (and should, I cannot stress its importance enough) build fully automated CI/CD pipelines for these branches, accompanied with a good hotfix tracking/management system in place, making the hotfix delivery just routine (or almost).

Improve this answer
1
  • Dan - so obvious and simple but it makes perfect sense. Money makes the world go round and in turn should help either compensate us for the cost of long lived branches or encourage customers to upgrade and stay close to trunk. Thanks for you good advice.
    – Mark Wheeler
    Commented Feb 22, 2018 at 16:07
1

Maybe if you maintained branches per versions instead of per customers it could help reduce their number?

Otherwise the only way to really get away from it is to be able to host the software yourself and switch to a SaaS model where you would be able to maintain only one version of it.

Improve this answer
1
  • Unfortunately our customers often operate in very shut down environments because of the financial data they are working with, so a SaaS model wouldn't be acceptable.
    – Mark Wheeler
    Commented Feb 22, 2018 at 16:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.