One Jenkinsfile or multiple?

Question

So I have a Jenkinsfile defining a build pipeline and then a Jenkins job (not pipeline) with a very simple deployment script for our Docker stacks.

Seeing that Jenkinsfiles can become as complex and powerful as one's coding skills are (seeing that they can groovy fairly well), would I have everything in one Jenkinsfile or have multiple Jenkinsfiles?

Consider this theoretical process:

Development
> Build artefact > Build Docker image
> Deploy Docker container to testing > tests & QA
> deploy to staging > have client check it
> deploy to production

Would all of that go into one Jenkinsfile and run different steps of the pipeline according to user input like

"which stages should be performed?
Please specify:
Build artefact,
build Docker Image, ..."

This would result in a complex Jenkinsfile but you'd only have one single file.

Or should this process be split into multiple Jenkinsfiles with multiple Jenkins jobs/pipelines for each Jenkinsfile to do a part of the process?

Answer 1

It really comes down to personal preference.

One additional tool you might not be aware of are shared libraries for Pipeline. These allow you to quickly write custom Pipeline steps or factor out common Pipeline code without writing a Jenkins plugin in Java.

Between multiple Pipeline jobs and shared libraries, there are many ways to split up your job's code and there's no one right answer on how to do it. One suggestion I have is to determine what steps in your process are "atomic" - that is, what are the smallest group of steps that you envision admins/ops/devs/etc. might need to run on their own? Each "atomic unit" should then become its own job.

For instance, let's say you're automating a deploy process, and your process in a general sense looks like this:

• build
• deploy to dev
• deploy to prod

Then I would create three jobs, one for each bullet point. This gives you some nice advantages:

• You can re-run each job individually in case something fails instead of having to restart the entire process over from the very beginning. Deploy to prod failed? You only need to re-run the deploy to prod job. You don't need to rebuild or re-deploy to dev.

• Each stage in the process is separated out so you can see how often it succeeds or fail. Some stages in your process may be more robust or fragile than others; this kind of separation gives you insight into that.

• This level of abstraction makes it easy for non-ops people to perform ops tasks (if that is something you want/need at your org). When everything is monolithic, the only thing non-ops people can do is run the entire process from the beginning. When everything is split up into tiny jobs, you need intimate ops knowledge to know which pieces to run in what order. By separating your jobs into independent, easy-to-understand, appropriately bite-size chunks, a non-ops person only needs to press a single button to kick off an automated ops process.

Answer 2

Having been down a similar road, here are my suggestions:

Break your tasks apart by function; build is different to deploy, which is different to release. You can get more granular, but let's stick to the basics for now...

Secondly, if you want to consider who is permitted to do certain actions, you might break these functions apart by team (dev, test, release).

Build pipeline:

• This is probably your dev team.
• A single job to check out sources, compile and package.
• Perhaps run unit tests against this package
• Perhaps send the code to SonarQube for analysis
• If successful, publish this build artifact somewhere, like Artifactory/Nexus
• I automatically run this job whenever someone pushes to the repo
• Additionally, I have some rules which limits behaviour based on the branch (I don't really want to publish builds for every single feature branch, but always for release branches)

Deploy pipeline:

• This job should be be generic, and accept parameters that gives it context
• Are we deploying to a dev server? Or to Test/Pre-rel/Live?
• Create a deploy-dev job, and a deploy-prod job. Both use the same Jenkinsfile
• Who has permission to perform this action?
• What are we deploying? This is a dropdown of artifacts available in Artifactory.
• I use JobDSL to generate multiple jobs that use the same pipeline. One for dev/test/pre/live. You could manually create these jobs at first with appropriate deploy targets/users

The above lets me write highly targeted and specific pipelines, yet gives the teams the flexibility they need for day to day stuff. Dev and Test are self-service. Only the release team has access to deploy to pre-rel/live.

Other pipelines:

• Once that's working, you'll probably look at jobs that can bounce servers (self-service for teams) and promote builds from snapshot/staging/RC repos to gold repos in Nexus/Artifactory, etc.
• Are you using git-flow? How about jobs to create/finish release/feature branches, bump maven versions, tidy up stale branches, perform integration builds, etc. All good fun.

Hope this helps.

Answer 3

I recently built a build-test-publish-deploy-rollback pipeline and struggled with the question of one-or-many pipes. I decided on a single pipe to do all tasks, with the reason that it would be easy to go to one place, and tell the pipe what you want to do by entering parameters. It seemed like a good idea at the time, but I have smelled a few drawbacks since releasing it:

1. A success or failure of a given pipeline build doesn't mean the application code is necessarily bad. For example, we had a push to production fail due to a bug in the pipe code, but that doesn't mean the app build is bad. But the Build Status Badge shows Failed in our ReadMe. Which isn't true (the way we see it).
2. It is more complicated. There are several paths through the pipe, determined by the parameter values. More chance for error, harder to review, etc.
3. Blatantly breaks the principle of single responsibility, as well as the principle of least amazement. It would not necessarily do what one might expect based on some rules i'd built in, for the purposes of protecting the users. With a simpler pipe, there'd be less chance of error.
4. Our org isn't evolved to true Continuous Deploy yet, so there will always be a delay and a person pushing the "deploy" button, even to QA. We may move to CD into QA sooner rather than later, but we have to prove the safety and reliability of all of this before the higher-ups will be comfortable with CD into production.

All of which speaks to separate pipes with clearly delineated functions.

I am now building another pipeline for a service of the same type, and I plan to refactor to use separate pipes to build and deploy - same as @jayhendren listed above:

1. Build, Test (barely exists at this point), Publish Artifacts
   1. I am trying to build cookbooks to spin up temporary machines to deploy to give devs siloed test sandboxes.
2. Deploy to QA
3. Deploy to Prod

One thing worth mentioning is we are definitely using Multibranch pipes, at least for the build pipes, and Declarative syntax wherever possible. Thinking about it as I write this, the Build pipe can stay as it is now, as a multibranch pipeline job built off of the Jenkinsfile living in the app's repo, and building on any push to github. Then I can manually build additional pipelines for deploying - templated so they're generic, and only pass the unique variables to each one.

Update:

I want to come back to this and contradict myself. After more internal discussion on the design, and my own growth as a Jenkins user, I went back to a single pipe for all tasks.

The difference was using fewer parameters, and controlling the flow with the branch/PR that triggered the build. This way, although the tasks may vary, a pipe's output is based on its properties only.

Obviously this reduces the overall complexity by only having one pipeline, although the single pipeline's conditional checks may become more complex to handle all the conditions.

I should also add that doing the non-code design work like flow charts and diagrams helped expose the possibilities - going straight to code hid the big picture from me at first.