What we need to do is be able to find a consistent solution to create a single package (lets call it Master package) that contains multiple packages, check that these packages are compatible with each other and version them.
The packages that we need to ship together are many (15+) and based on different technologies where many of them depend on each other, so consolidate them in a single repo is not a possible solution.
We need to take in account the eventual limitations of the tool we are using for CI/CD/Artifacts that is Azure Devops and Yaml pipelines.
At the moment we are producing a Universal Package for each component to be released, and we are versioning them using gitflow and semver (x.y.z version).
The main problem now is trying to find a way to put all these pieces together in a simple way and manage the compatibility of these pieces.
The distribution of these packages is out of the scope of this question, we only need to create the package containing everything, the installation/distribution process is already in place.
What we are doing now
At the moment the process is this.
For each component we are creating a Universal Package with an x.y.z version.
When we want to create a Master package (also a Universal Package) we have this pipeline on Azure Devops, which parameters are all the versions of each component, that by default download the latest version of a package (using '*' wildcard, so it doesn't support versions containing dashes, only x.y.z). Using latest as default parameter make the pipeline easy to run for everyone.
The problem with this approach is that:
Azure Devops pipelines parameters don't offer a way to display a dropdown list of available version for each package, so having to open Azure Artifacts feed every time we need to run a build, to choose a certain version is not a viable solution We have no way to check if components are compatible with each other, having to hope that all the latest versions can work together A way to handle the first problem that we implemented, is to create a release branch on the repo that contains the pipeline definition and use fixed version as default parameters of the pipeline, but if we need to bump a version we still need to make a new commit and requires still a lot of manual handling.
Some solution that I thought
A way to version the Master package could be using CalVer YYYY.MM.Build, given that is a standalone system.
To solve the problem of compatibility we could write a manifest file for each component and define inside of it a constraint on the version that requires from other components.
Example: If component C depends on B and B depends on A In the manifest file of C we say that B must be 1.*, and in the manifest of B we say that A must be 2.0.0
But in this solution, we may need to make commit and a new release of C or B only to bump the accepted versions.
To solve the problem of user experience we could create an external portal, in this way we could create some kind of dropdown list to make the version selection easier (is there already a solution on the market to solve this problem?). In this solution we could also integrate a check on the manifest file, so we can see beforehand if a Master package is consistent within itself.
Another solution that I thought, to solve the user experience problem, could be create a separate repo, where we write a manifest file in which we define all the packages and their version to pack together, and in some way pass it to the pipeline. In this way we resolve the problem of having to check foreach run the azure artifacts feed, but we still need to do a lot of manual handling, and it doesn't offer a solution to check the compatibility of packages (beside implementing it in the pipeline itself).