Is it a good practice to run unit tests inside build job in Gitlab CI?

Question

• We want to run unit tests against a built Docker image and not by installing the environment inside Gitlab CI runner
• We don't want to push the docker image to a registry and pull it back in a test job to run unit tests - it's slower.

But it contradicts the separation of concerns principle - now our build job will also run unit tests.

The question for the community: is it okay to sacrifice this principle in this case or there are better alternatives?

Answer 1

TL;DR: See bold sentence below.

---

I know the separation of concerns (SoC) principle mostly from a "software code organisation" point of view. I never explicitly encountered it for build job definitions up to now (which does not mean it is bad to follow it in this context too).

From my point of view: Principles and good (or even best) practices exist for a reason. Following them can help you guide your project to success, because they seem to appear often in successful projects^{[citation needed]}.

This does not mean that every successful project follows all best practices. On the other hand, following them does not automatically make your project successful.^{[citation needed, too]}

Please note that I don't know building Docker images with GitLab well enough to give you advice in this direction. Maybe GitLab provides some special mechanism directly for this use case?

---

In your case:

The main goals of SoC are to manage complexity, increase modularity and facilitate maintenance.

I summarise the points you give in your question as increase pipeline efficiency.

So the questions are:

• Does "test during build" increase the performance so much that it pays the additional maintenance costs?
• How complex are the building and testing logics for themselves? How easy is it to split them apart again if you put them together now?
• To put in the two additional principles "Premature Optimization" and "You Aren’t Gonna Need It": Do you really need this efficiency now? On the other hand, do you really need the separation now?
• Maybe it is possible to do both, separate building and testing each in their own script files but call them both from one job?

If you cannot find strong arguments in favour of one or the other now, do as you like and be prepared to refactor later when your needs become clearer. Whatever you do, document your decision and the reasons for it together with the alternative as comments in your pipeline definition. Example:

Build and test are done in a single job to avoid pushing/pulling the Docker image to/from registry to increase performance. If the complexity of build/test logic increases too much, split them in separate steps and check if performance is still acceptable.

This way you and your colleagues in the future can better understand why it is the way it is currently and whether it makes sense to change it.