Shared Jenkinsfile for multiple pipelines (separate source repositories)

Question

I'd like to write one Jenkinsfile to be shared between a number (~50) of pipelines/jobs, where each job handles the creation of a library from a specific source (git) repository.

Currently I have one job which uses SCons and build parameters to specify/calculate which repositories to build, but this involves cloning ~50 repositories each time and means features like the git change notes don't work (the repository with changes that is built isn't the repository containing the builder code, which typically just shows "No Changes").

Is there a better way to handle this situation? I see in this question (Modularizing CI/CD pipelines in Jenkins or in other tools?) that shared libraries are the suggested solution there, but whilst I can see that this simplifies the N Jenkinsfiles, it doesn't as far as I can tell reduce their number.

I'd like to avoid having any Jenkinsfiles in the source code repositories if possible (perhaps instead having information in the job configuration that boils down to "check out and run this Jenkinsfile from that other repository, as if it were in this one").

Answer 1

Putting Jenkinsfiles in every single repo is, as far as I can tell, unavoidable. As you mentioned in your question, you can use shared libraries to centralize the Pipeline code itself, but not the actual Jenkinsfiles, which will need to load/execute the shared library code.

The way I've dealt with this is to use external tools to enforce that all repositories contain a copy of a Jenkinsfile (which does nothing but load and execute the shared library code). For instance, our Terraform plan creates repositories from a template, and the template contains the aforementioned Jenkinsfile. We also use a tool called Thor during one of our build steps, and one of the tasks Thor performs is to synchronize certain files (including the Jenkinsfile, but also other files like licenses) to match a copy from a central repo. So whether I am creating a brand new repo or making changes to an existing one, there is always a tool to ensure that a simple Jenkinsfile is in place.

Answer 2

Try to put your base Jenkinsfile into Jenkins Shared Library. You can also make it parametric as a regular function. Please check out the following official document's related topic:

https://www.jenkins.io/doc/book/pipeline/shared-libraries/#defining-declarative-pipelines