Trying to migrate CI/CD process from Jenkins to Gitlab Runner + Docker on Windows

Question

I need to improve the CI process of my projects, which is basically composed of the following things:

• a Linux server, that contains a Jenkins master
• a VM with Windows 10, with a Jenkins slave, responsible to build one of my projects (that uses .NET core, .NET framework, Node.JS and nmake)
• a Jenkins pipeline job that compiles the project (assuming the needed tools are already installed) -> this is used to generate final released versions to the customer
• a Gitlab Runner on some Windows machines (configured as 'shell' executors) -> this is seamlessly used with our gitlab, for continuous integration and pipelines, to be visible on Merge Requests, for instance

Problems: we manually entered the VM above and installed all the tools needed to build the applications. If the projects require new tools, we need to manually enter the machines and install them prior to running the build jobs.

What I want: I plan to improve this CI environment with Docker (however I'm pretty new to it), to be able to easily scale the job runners. I'm thinking about using less Jenkins and more Gitlab Runner instances (configured as 'docker' executor), to be able to delegate the CI/CD processes to each '.yml' file versioned on each project.

To accomplish the above, I thought about some approaches:

Put a Gitlab Runner configured as 'docker+machine'

This spawns as many 'docker' runners as needed --> this is easily scalable

Problem: "Docker Machine" only creates Linux VMs, and I need specifically for this project a Windows environment for the build

Use a Gitlab Runner configured as 'docker'

This should be scalable, if it spawns containers for building on demand.

Based on this scenario, I have a couple of questions:

1. do you have any suggestion about how can I create a docker image that is able to build my project? I need to use Windows OS, .NET core, .NET fw, nmake, Node.Js. Do you have some specific images recommended for this task?
2. should I create a 'sdk' docker image (an image with all tools needed for building and testing) separate from my project source-code, and publish this image to Docker Hub, and 'consume' it on my gitlab runner? I mean, what's the best practice for this case, supposing my customer does not have a private Docker Registry?
3. have anyone tried this scenario (gitlab runner + docker on Windows) before, and have some feedback if it works or not?
4. Is my plan of decreasing Jenkins usage and using more Gitlab runner (for this project at least) feasible? I thought about this because we currently have to manage a "Jenkins Master" and all "Jenkins Slaves" machines. With this new approach, the 'master' would be actually the entire Gitlab instance, and the 'slaves' would be several GL runners deployed as I need, on which I would delegate the job to each project's ".yml" file;
5. Do you have a better approach on how can I improve to a scalable Windows project CI/CD process, other than using gitlab runner + docker?

Answer 1

have anyone tried this scenario (gitlab runner + docker on Windows) before, and have some feedback if it works or not?

I installed and registered a GitLab Runner on my Windows 10 laptop following the instructions from GitLab Install GitLab Runner on Windows

At first, I had trouble pulling the Windows container image:

PS C:\GitLab-Runner> docker pull mcr.microsoft.com/windows/servercore
Using default tag: latest
Error response from daemon: manifest for mcr.microsoft.com/windows/servercore:latest not found: manifest unknown: manifest unknown
PS C:\GitLab-Runner> docker pull mcr.microsoft.com/windows/servercore:1809
1809: Pulling from windows/servercore
no matching manifest for linux/amd64 in the manifest list entries
PS C:\GitLab-Runner>

After selecting the "Switch to Windows containers" option in my Windows Docker (in the top-level menu that opens from the Docker icon in the toolbar), I was able to docker pull the Windows servercore image.

However, I was unable to run GitLab Runner on my fully patched and updated Windows laptop:

 ERROR: Preparation failed: unsupported Windows version: Windows 10 Pro Version 1909 ...

It's too new! As I read GitLab's Windows version support policy, GitLab should support 1909 for 18 months; but obviously that is not the case yet. 1903 is the most recent supported version.

However, as of January 2020, GitLab has Windows Shared Runners (beta)

I tried that and it works. Example .gitlab-ci.yml:

my_job:
  script: systeminfo
  tags:
    - windows

Is my plan of decreasing Jenkins usage and using more Gitlab runner (for this project at least) feasible?

Sounds good to me.

Do you have a better approach on how can I improve to a scalable Windows project CI/CD process, other than using gitlab runner + docker?

No. I think you're on the right path.