I need to improve the CI process of my projects, which is basically composed of the following things:
- a Linux server, that contains a Jenkins master
- a VM with Windows 10, with a Jenkins slave, responsible to build one of my projects (that uses .NET core, .NET framework, Node.JS and nmake)
- a Jenkins pipeline job that compiles the project (assuming the needed tools are already installed) -> this is used to generate final released versions to the customer
- a Gitlab Runner on some Windows machines (configured as 'shell' executors) -> this is seamlessly used with our gitlab, for continuous integration and pipelines, to be visible on Merge Requests, for instance
Problems: we manually entered the VM above and installed all the tools needed to build the applications. If the projects require new tools, we need to manually enter the machines and install them prior to running the build jobs.
What I want: I plan to improve this CI environment with Docker (however I'm pretty new to it), to be able to easily scale the job runners. I'm thinking about using less Jenkins and more Gitlab Runner instances (configured as 'docker' executor), to be able to delegate the CI/CD processes to each '.yml' file versioned on each project.
To accomplish the above, I thought about some approaches:
Put a Gitlab Runner configured as 'docker+machine'
This spawns as many 'docker' runners as needed --> this is easily scalable
Problem: "Docker Machine" only creates Linux VMs, and I need specifically for this project a Windows environment for the build
Use a Gitlab Runner configured as 'docker'
This should be scalable, if it spawns containers for building on demand.
Based on this scenario, I have a couple of questions:
- do you have any suggestion about how can I create a docker image that is able to build my project? I need to use Windows OS, .NET core, .NET fw, nmake, Node.Js. Do you have some specific images recommended for this task?
- should I create a 'sdk' docker image (an image with all tools needed for building and testing) separate from my project source-code, and publish this image to Docker Hub, and 'consume' it on my gitlab runner? I mean, what's the best practice for this case, supposing my customer does not have a private Docker Registry?
- have anyone tried this scenario (gitlab runner + docker on Windows) before, and have some feedback if it works or not?
- Is my plan of decreasing Jenkins usage and using more Gitlab runner (for this project at least) feasible? I thought about this because we currently have to manage a "Jenkins Master" and all "Jenkins Slaves" machines. With this new approach, the 'master' would be actually the entire Gitlab instance, and the 'slaves' would be several GL runners deployed as I need, on which I would delegate the job to each project's ".yml" file;
- Do you have a better approach on how can I improve to a scalable Windows project CI/CD process, other than using gitlab runner + docker?