Sorting out automated testing in CI pipeline and getting conflicting information related to building, unit testing and Docker images

Question

I seem to get a unique response for every person that I ask, or every article that I read, which is just confusing me even more.

Basically, I'm trying to setup automated testing in the CI pipeline which I'm pretty new to. The stack is basically these things:

• Django API service
• React Admin Portal service
• React Client Portal service
• PostgreSQL
• Redis
• Azure Kubernetes Services
• Azure DevOps Pipelines with Microsoft Hosted Agent

I was originally under the impression thanks to advice here, which was pretty consistent among respondents, that basically my CI pipeline should be the following:

1. Build the services.
2. Run unit tests.
3. If passing, COPY the build into the Docker image.
4. Push Docker image to Container Registry.
5. Deploy images to production-like Kubernetes (using Kind, Minikube, etc.) in a pipeline VM ("Hosted Agent" in Azure DevOps terms).
6. Run integration tests.
7. Run E2E/Selenium tests.
8. If passing, merge feature branch to production.

Started talking to someone else about it and they basically said:

Unit > integration > e2e. Test as much as you can at each step without pissing the devs off. Testing in parallel is better because it lessens the time taken. If a failure in any test type means the pipeline breaks the only thing that controls when you run the test is if you can do it before the app is running (unit, functional) or after compile and deployment (integration/e2e).

All of that aside: DON'T COPY COMPILED CODE INTO A CONTAINER. Such an anti-pattern and bad practice. The Dockerfile should include the entire build process and remain immutable once built. Copying code into your container is the equivalent of passing it to a coworker and assuming it's going to run on their laptop.

Multi-stage builds and development containers containing sdks/build-kits/whatever are the first step, runtime containers that contain only the necessary pieces to run are the second. If they aren't available from the platform provider - .net, java, python etc. (they are) - you can build them both out of the same base container which standardizes both the build and runtime environments.

Use multi-stage Dockerfiles and combine to reduce steps so that containers stay lean.

In other words:

1. Build the services in a Docker image.
2. Run unit tests in the the Docker image.
3. If passing, deploy images to production-like Kubernetes (using Kind, Minikube, etc.) in a pipeline VM ("Hosted Agent" in Azure DevOps terms).
4. Run integration tests.
5. Run E2E/Selenium tests.
6. If passing, push to Container Registry and merge feature branch to production.

Which, unless I'm misinterpreting this, contradicts some other statements I've received like this:

Never, ever build in docker images, because it means we have to install dev-kits in the container. No docker container in production should have any of these kits. For security.

I can see the value of both approaches.

I'm basically just looking for more clarification on this process regarding building, unit testing and Docker images. Suggestions?

Answer 1

You've missed the part of the 1st quote about multi-stage builds and other methods which avoid exactly the problem mentioned in the 2nd quote: the resulting image would consist of it's base image and artifacts from previous stages while being run in the same build step of a pipeline.