Combining CD and GitFlow

Question

Firstly, I apologize if the title is not descriptive enough or does not describe my problem well enough. I'm struggling to figure out how to describe my issue in brief! It really seems like this should be a problem that has been solved already, but I'm struggling to find the answer. I might be missing something glaringly obvious!!

So - we currently have a Git workflow on GitLab using 3 core branches (master, preprod and develop). We branch features off from develop, then back into develop for initial testing. Merging into develop triggers a pipeline that tests, builds and deploys to a dev server, so that Project Owners can review code in progress. Once signed off, the develop branch gets merged to preprod, which also triggers a pipeline to deploy to our UAT environment for... UAT. Once signed off, preprop gets merged to master which has a manually triggered pipeline to deploy to live.

Bunch of issues with this approach, but the biggest is the bottle neck. If we have a feature in preprod that we don't want to release yet, but another feature is coming up behind it, it's tricky to bypass the feature we don't yet want. So we are considering moving to a different branching strategy such as GitFlow.

Now on to the bit I can't figure out! Currently, in order to deploy a feature to an environment where it can be viewed and tested, we have pipelines triggered by a merge to specific branches. However, I would like to do away with this approach when testing features and hotfixes. Essentially, I need a way to be able to run a pipeline that will run automated tests, build and deploy any branch at any time. So if I am working on FeatureA, I can push that branch to GitLab and deploy it somewhere so that it can be reviewed in isolation from FeatureB that is being worked on by a colleague. Does that make sense? Almost like a dynamic pipeline with a dynamic environment that spins up when needed and gets destroyed again when no longer needed.

If anyone can point me in the right direction here, it would be appreciated!

Should also add that we are starting to use docker on our local machines for development, so a solution that uses docker for spinning up dynamic test environments is not out of the question.

Thanks in advance...

Answer 1

Conflating deployment environments and branches often leads to problems like you describe. It does seem like a natural fit though and easily maps to mental model of an ideal solution but, real life is anything but ideal.

Basically, git is not a deployment tool, one should not clone to deploy a version but fetch a package (a tarball, zip file, docker container, NPM package etc).

In practice these two concepts are orthogonal. Branching strategies (whichever it may be, gitflow, trunk based etc) is a means to assist developers isolating and sharing their work. It ties naturally to the versioning strategy and is what forms the CI half of your CI/CD system. Here you must force yourself to keep CI (continuous integration) separate from CD (continuous deployment) as you will see if you read-on.

Deployment however consumes the outcome of versions that were created and integrates various parts of the system and makes a whole that is usable, testable.

So if you view your branches and the code therein as a means to assemble a version which the build system will build and package and then store it somewhere. Can be as simple as a network share or FTP site or it could tie in to an existing package manager with a appropriate back end, whatever suits your team best really. This choice will depends greatly on the technology stack you are using and the underlying platforms you will be deploying to (static environments, docker, kubernetes, standalone copy-and-run etc).

Then another set of tasks can pull in any of these versions and assemble a working deployment. Here this works best if your deployment environment are dynamic but it can be made to work with static environments as well. This will form the CD part of your continuous system.

Once decoupled you can spin up an environment that would contain any versions that has been previously built.

In term you could even have multiple production environments with slight variations of a user interface tweaks and pit them against each-other with techniques like A/B testing and clever load balancing / distribution to end-users.

Going back to your particular situation it will simplify your gitflow back to a more traditional master/develop permanent branches and a set of features/hotfix and release branches.

Using semver you can keep official versions strictly for master merges (from release branches or hotfix branches) and pre-release versions for all other branches. for your pre-release branches simply add a -SOMETHING where something is the name of the branch for example and possibly a timestamp to differentiate successive deployments.

for example,

• release branches would get the x.y.0-RCi (i th release candidate). x.y is the version that will be released where the -RC is simply dropped when merged in master. 0 as the patch because release branches are used for new features and product evolution
• hotfix branch follow the same principle as the release branch except the patch number is set to a non-zero value.
• development branch could get x.y.z-dev.timestamp
• feature branch x.y.z-JiraID-timestamp. Where JiraID is the ticket number of the system you use to track new features (like Jira, Bugzilla, etc)

I propose semantic versioning here because I am familiar with it but the same broad principle of segregating branches through versioning can be applied with pretty much any versioning scheme.

Nowhere in the source control do you see anything that relates to deployment environment. Only your versioning strategy is what links them, they are otherwise completely isolated and independent.

Your devs will feel more comfortable in this new environment and your deployments will feel less constrained and awkward.

Hope this helps