It is a common scenario that the codebase of a product held by a repository in some VCS system evolves to a point where that codebase can arguably be seen as containing several products. Splitting the codebase across several VCS repositories, each dedicated to a single product, can leverage several benefits (see Benefits of having a product per VCS repository over the bloat repository model below). On the technical side, splitting the codebase is a rather easy step as most VCS support this operation. The split however might rise engineering issues related to automated testing, continuous delivery, service integration or monitoring (see Issues raised by the split.) Organisations planning to perform such a split therefore need to know how to perform this transition as smoothly as possible, that is, without interrupting their delivery and monitoring pipeline. The first step of this is probably to better understand the notion of project and how to delineate the split in a monolithic codebase.
In the answers to this questions, I would like to see:
An attempt to give a working definition of what a product is, which gives practical criterions to actually delineate products in an existing codebase.
According to this working definition, elaborate a plan that actually perform the split. We can make the simplifying assumption that the codebase is processed by a fully automated sdlc implementing continuous-integration and continuous-delivery. That is, each branch is validated by an automated testsuite implemented in the current codebase and each merge to some “magic” branch generate product-artefacts that are tested and deployed. (Product artefacts are e.g. source tarballs, documentation, binary software packages, Docker images, AMIs, unikernels.)
Such a plan is satisfying if it explains how to circumvent the
Issues raised by the split
How automated testing procedures relate to the pre-existing monolithic repository and the split repositories?
How automated deployment procedures relate to the pre-existing monolithic repository and the split repositories?
Where is stored the code for automated deployment procedures themselves?
Where are stored infrastructure, monitoring and high-availability strategies?
How to ensure that a developer needs only one codebase at a time (but possible uses artefacts from other codebases).
How can a tool like git-bisect
Marginal note: Benefits of having a product per VCS repository over the bloat repository model
Having several small repositories holding the codebase for a specific product has the following advantages over the “bloat repository” approach:
With a bloat repository it is hard to roll back a release when a product is unstable, because history is mixed with other product history.
With a bloat repository, it is hard to review project history or pulls, with small repositories, we are more likely to read this information. (This might be specific to VCS like git, where unlike svn, we cannot checkout subtrees!)
With a bloat repository, we have to do much more branch-dance when we develop. If we have N repositories we can work in parallel on N branches, if we have only 1 repository we only can work on one branch, or have a load of working copies which also are a hassle to handle.
With several small repositories, the logs give a heat map of the project. They can even be used as a proxy of knowledge diffusion in the dev team: if I did not commit in repo X since 3 months, it could be good to assign me in a team working on repo X so that I stay aware of the developments in that component.
With small repositories, it is easier to gain a clear overview of a component. If everything goes in a single big repository, there is no tangible artefact delineating each component, and the codebase can easily drift towards the big ball of mud.
Small repositories force us to work on interfaces between components. But since we want to have a good capsulation, this is a work we should do anyway, so I would count this as an advantage for small repositories.
With several small repositories, it is easier to have several product owners.
With several small repositories, it is easier to have simple code standards that are pertinent to a full repository and that can be automatically verified.