How is it a good practice to version your helm charts for each deployment?

Question

I'm trying to set up a process to release a web app with helm in my company and I can't find a satisfying way.

Basically each for each push in a feature branch, I want to:

1. Build a docker image and push it to a registry
2. Upgrade the app through helm

Step 1 is easy, we keep the Dockerfile in the sources, which makes sense because it allows developers to test the docker image and the CI can also build the image and push it automatically.

Step 2: I'm not sure. I was told it could be interesting to keep track of the deployment history and therefore to update the helm chart version for each deployment which seems do-able but I'm not sure I see the point because helm history already provides the info I guess?

Also it forces the CI to actually commit and push the new chart version itself!

My idea was to keep the helm chart in a separate repository with all our charts and to just use keep track of the changes of the chart but nothing to do with any deployments so: the CI could basically build the docker image, push it, clone the helm repository and do the following:

helm upgrade $APP_NAME --set image.tag=$APP_VERSION ./path/to/chart

(with APP_VERSION being auto generated and matching the docker build tag)

I haven't seen this approach when reading about it, most people seems to keep their app version up to date in their helm charts and I'm wondering how is that supposed to work for continuous deployment?

Answer 1

Approach you are describing - where app version is substituted at helm upgrade command - is pretty popular as well. Maybe it's not well documented - I'm not sure about that.

Yet another approach that people use is doing helm template which would compile everything (including variable resolution) into fixed k8s definitions and then doing GitOps on that.

There is no universally accepted approach here among those 3 (1st being updating helm chart itself with new versions every time). So you should decide what works better for your use case.

One thing that is worth mentioning is versioning issue (disclaimer: I'm solving it with Reliza Hub, which I develop). Basically, you need some system which would tell you what version tag of each microservice to actually use on different environments. That problem may become quite complex, especially if you have many microservices and many environments.

Answer 2

So, I actually ended up with the following strategy.

I separated the charts from the source code. Basically, I have a repository that tracks all the helm charts only

Why?

1. I wanted to have a repository representing my cluster(s) so it's easy to rebuild a cluster from it
2. I also wanted to keep track of the current version deployed. Basically, this repository is the truth of what is exactly deployed on the cluster(s) knowing that it tracks the tag of the docker image that is used for each app, so if I want to go back in time, I could just revert a commit or update the docker tag for one of the apps in this repo
3. Separation of concerns: developers don't have to care how the app is deployed, they basically only care about how to build a docker image that works

How does that work?

It depends on the project, but for our API server for ex, we have 3 version online: devel.api.mycompany.com staging.api.mycompany.com api.mycompany.com

1. a PR is validated and merged on the app repository
2. The app CD builds a docker image with a tag = current git sha and pushes it to our private docker registry
3. The app CD clones our charts repository and update the docker tag version for this app and pushes on the repo
4. The charts repo receives a push with a commit updating the docker tag of the app
5. The charts CD runs helm upgrade for this app, which deploys the new pod on the cluster (so using the new docker image version)

Note: We use helmfile on the charts repository, so it's easy to update by cluster and by app or all together + runs lint etc etc.

Not saying it's the best but we're currently satisfied so hope that helps.