How can I unify my distributed build environments using Jenkins and Docker?

Question

I have recently taken over the CI management for a medium-sized team of developers. Currently, we have one Jenkins server instance as well as six other instances of Jenkins agents to build upon. We have been using labels to determine which builds to run. For example, some instances are setup to run under environments A or B. In the future, we would like to add more instances of A and B, as well as add new environments, e.g., C, D.

My goal now is to unify these build environments and define them in one place to allow for better scale-ability and ease of management of these instances. Docker seems to be a solution to this approach. I know that I can create a Docker image for each of my individual environments. What I don't know is how to distribute the image into containers across my six machine instances. The solution I am looking for involves allowing multiple docker containers to be spun up on a single instance (docker host?), but only if needed (i.e., the scenario when all other hosts are currently hosting a container). Additionally, I need a solution that supports Windows and Linux based hosts.

I know that Docker support in Jenkins seems to be based largely via plugins, so I was hoping for some direction in which plugin to research first.

Answer 1

It sounds like you have a main Jenkins orchestration server and six additional nodes added for build capacity. If you are attempting to improve scalability, there are quite a few options.

If you have containers representing the build environments (A, B, C D), an option would be docker swarm across the existing machines, allowing all of the hosts to be managed as a group and your containers to be dropped on them. https://github.com/jenkinsci/docker-swarm-plugin

Similarly, but more complicated, create a kubernetes cluster (Openshift, Rancher, etc.) with your current six machines and allow the main Jenkins to spin up and use containers on the clustered kubernetes as needed. This would allow you to create, add additional machines to the cluster, if scaling is needed. You could even add your main Jenkins server to the cluster and run Jenkins as a container. https://github.com/jenkinsci/kubernetes-plugin

Another option is to add cloud (GCP, Azure, AWS) capacity as a node(s) to push overflow to pay as you need capacity in the cloud. https://github.com/jenkinsci/google-compute-engine-plugin

Lastly, you could also use your current solution, as is, while developing a cloud native CI/CD pipeline and slowly migrate to a more modular, scaling infrastructure. There are so many options in this space, some not even needing Jenkins, solutions could be tailored to specific needs.

I hope this gives you some ideas and hints!