Jenkins pipeline running using containers on an orchestration platform

Question

I have a specific implementation of jenkins on docker.

My requirement is to start slave containers on demand and only when they are needed and then stop them after the pipeline has ran.

I have the jenkins master instantiated as a container, then using the docker socket I am able to run my pipeline in one or more other containers containers provisioned by jenkins, that I have specifically built for that purpose following the implementation described in the jenkins book.

• master container - https://github.com/YiannisH/containerised-jenkins/blob/master/docker/master/Dockerfile
• slave container - https://github.com/YiannisH/containerised-jenkins/blob/master/docker/slave/Dockerfile

The above mentioned implementation is implemented on one EC2 machine acting as a docker host.

However, that implementation has the flaw that is not highly available and I am only able to use the resources available on on server.

How can I implement this on a swarm cluster where I still have just one master jenkins container starting slave containers from the pipeline on a swarm cluster?

I cannot seem to find something similar after extensive searching, but I am sure it is in some way possible.

Answer 1

You can run your application on a particular node of the cluster by defining it in the application level. For this, you need to add the "placement " parameter in the yaml file and specify the node's name to it, then it will deploy the container on that node.

Secondly, what type of scheduling have you implemented on Swarm CLuster? Let me clear your thought, there three of them: Spread, Bin-pack, and Random. By default, it is Spread, which identifies a number of containers on each node and deploys the new container on the node that has a small number of containers. Check the rest yourself.

Both of these points will solve your problem.

Answer 2

You can use the docker-swarm-plugin.