On Premise CI / CD recommendations for Docker images

Question

We have TFS building Docker images, and loading them into a Private Registry on one network, our Development Network. Everything works well, and we are generally happy with the CI process in that environment. We have an Air-gapped Integration Network that we need to export these Docker images to, and then have something that finds them, puts them into a registry there, and then restarts a Docker swarm.

I am looking for a solution that can:

1. Watch a folder for new files
2. When a new Docker Tar file is found, load it into Docker Registry.
3. Move the Docker image that was loaded to an "archive" folder
4. Restart Docker swarm to use new images in the Registry

It seems like step 2 might not be possible directly, I might need to load it into a local Docker instance, tag it, and then push it, but if so, that's fine.

Is this something I should Jenkins for? I could just write a script in Bash or Python, but I would like to use a proper CI tool, if I can.

Answer 1

Jenkins Option

If you're comfortable with Jenkins and want to use a CI tool, it is a decent option.

It has multiple plugins for watching for files arriving in a file system, and at the end of the day, you can even just wrap your bash scripts into it so you have a central GUI and recorded logs. So, you're not limited in any way by it.

Inotify Option

I personally might just have a CI tool like Jenkins or GitLab-CI deploy a bash script though. Then you could use inotify to properly watch a folder and execute your script exactly when things arrive. See: https://www.howtogeek.com/405468/how-to-perform-a-task-when-a-new-file-is-added-to-a-directory-in-linux/ for an example.

Why iNotify?

In many tools, I believe you generally have to "poll", which means run over and over just to see if you have anything to do. Inotify lets you genuinely "watch" and be pushed events only when things happen. This is cleaner and more efficient as you won't have hundreds or thousands of tasks running a day that literally do nothing.

Monitoring

Note: One plus side to the pure Jenkins approachis built in monitoring. Every job run that fails can easily email/alert you. You can pull this off with the iNotify script as well, but if you don't have the right tooling for monitoring/etc in your environment, you may favor the Jenkins approach.