I have a specific implementation of jenkins on docker.
My requirement is to start slave containers on demand and only when they are needed and then stop them after the pipeline has ran.
I have the jenkins master instantiated as a container, then using the docker socket I am able to run my pipeline in one or more other containers containers provisioned by jenkins, that I have specifically built for that purpose following the implementation described in the jenkins book.
- master container - https://github.com/YiannisH/containerised-jenkins/blob/master/docker/master/Dockerfile
- slave container - https://github.com/YiannisH/containerised-jenkins/blob/master/docker/slave/Dockerfile
The above mentioned implementation is implemented on one EC2 machine acting as a docker host.
However, that implementation has the flaw that is not highly available and I am only able to use the resources available on on server.
How can I implement this on a swarm cluster where I still have just one master jenkins container starting slave containers from the pipeline on a swarm cluster?
I cannot seem to find something similar after extensive searching, but I am sure it is in some way possible.