17

This is a follow up question to my post Parameterized Kubernetes manifest files?.

I understand more about Helm now, after lots of reading, including the Learn Helm book (which I recommend). I still have this understanding that each microservice will have its OWN Helm chart? This defeats the purpose of what I wanted, where I just have ONE helm chart, and every time I want to use it, I just update the Chart.yml and values.yml files for each microservice, everything else is the same. For example, the microservice name, versions, and repo will change of course per microservice. Seems like I'm looking for a "template" for Chart.yaml and values.yaml. Is this possible to do? I don't know if it's recommended or not (I may not care).

I come from a CloudFoormation background, where I just have ONE parameterized CloudFormation template, and I just pass it parameters pertinent to each microservice.

UPDATE: I'm aware of Helm library charts, and I don't think they can provide the functionality I want?

4
  • What problem are you trying to solve? Are there too many lines of yaml? You can do as do describe, but it is not a good practice.
    – Jonas
    Commented Feb 17, 2021 at 20:18
  • 1
    Hi again @Jonas. ALL my microservices (will) use the SAME set of charts. They will literally only differ in name, version, ingress path. If I had one Helm chart per microservice, which I think is the recommended way to do it, I will just literally copy/paste the chart files from one microservice to the other, all templates files will be the SAME, and I'll make changes in Chart.yaml (name, version, etc.) and values.yaml (image, ingress, etc.). So if I update a templates file in one, I'll have to do it in all. What's the point of doing all this?
    – Chris F
    Commented Feb 17, 2021 at 20:46
  • It sounds like all applications is managed centrally in your org? Usually, different teams manage their own apps.
    – Jonas
    Commented Feb 17, 2021 at 21:09
  • Yeah, I manage ALL of them, or at least their deployments. LOL.
    – Chris F
    Commented Feb 17, 2021 at 21:10

4 Answers 4

18

Subcharts are the way to go.

I've done a very similar thing as follows:

  • Create one or more base charts which cover microservices that have similar configuration (e.g. one for backend microservices, another for frontend servers).
  • Create an overall chart for your application. This may be a bare bones chart without any resource configurations, but just a Chart.yaml and values.yaml. Or, you may also include some resources specific to your application overall and not a particular microservice (e.g. networkpolicy or ingress).

You could put everything in one git repo e.g.:

my-deployment-repo/
|- base-microservice/
   |- templates/
      |- deployment.yml
      |- service.yml
   |- Chart.yaml
   |- values.yaml
|- base-ui/
   |- templates/
      |- deployment.yml
      |- service.yml
   |- Chart.yaml
   |- values.yaml
|- myapp/
   |- Chart.yaml
   |- values.yaml

Then, your overall application chart (here myapp) can include the base charts multiple times as a dependency in its Chart.yaml. If you are putting all these charts in one git repo, you can use relative paths to point to them. E.g.

# Chart.yaml
dependencies:
  - alias: my-microservice-1
    name: base-microservice
    version: "0.1.0"
    repository: file://../base-microservice
  - alias: my-microservice-2
    name: base-microservice
    version: "0.1.0"
    repository: file://../base-microservice
  - alias: my-ui-1
    name: base-ui
    version: "0.1.0"
    repository: file://../base-ui
  - alias: my-ui-2
    name: base-ui
    version: "0.1.0"
    repository: file://../base-ui

The key thing here is alias. This allows charts to be depended on multiple times. It sets the name of the chart to the alias, so within the subcharts you can use {{.Chart.Name}}.

Finally, in myapp's values.yaml, you can pass different values to the subcharts under their alias key, e.g. if you have used {{.Values.image}} in base-microservice, you can do the following in myapp's values.yaml:

# values.yaml
my-microservice-1:
  image: foo

my-microservice-2:
  image: bar

When it comes to deploying myapp, run the following commands from within the myapp directory:

helm dependency update
helm install myapp .

You must always run helm dependency update before installing/upgrading a chart if any of the subcharts have changed.

I recommend git ignoring .tgz files, so you don't end up with .tgz files created by helm dependency update getting committed to your repo.

9
  • I love this approach but sadly there are Helm bugs that prevent it from working right. You cannot reference the aliased names in the parent chart in various contexts. (github.com/helm/helm/issues/7729, github.com/helm/helm/issues/2993, github.com/helm/helm/issues/7093, github.com/helm/helm/issues/3457)
    – TrogDor
    Commented Oct 14, 2022 at 17:56
  • I'm FINALLY getting back into this project (LOL) so I'll try this approach. Thanks!
    – Chris F
    Commented Jan 24, 2023 at 17:39
  • So in the myapp/templates/deployment.yaml for example, how would you reference the image.name for each alias for my-microservice-1? Will I just do a range here, and it'll install ALL the my-microservice-*?
    – Chris F
    Commented Jan 24, 2023 at 18:23
  • Ignore my last comment. I should mess with the base-microservice templates. Should I even have myapp/templates/ ?
    – Chris F
    Commented Jan 24, 2023 at 18:49
  • So with my idea, you wouldn't have a templates folder in myapp (unless you wanted some resource for the overall app and not for every microservice). The Chart.yaml pulls in the base charts multiple times as dependencies and values.yaml configures them.
    – emorris
    Commented Jan 25, 2023 at 22:36
8

Generally, a Helm chart would cover all your microservices - ideally, you should be able to deploy whole application in one chart. It may get a little messy if you include external 3rd party dependencies, where you would need to handle dependent charts, but I assume this is out of scope of your question.

More so, if your microservices are very similar you could write for loops, such as:

{{- range $index, $service := .Values.myservices}}
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-{{ $service.name }}
# --- etc ---
{{- end }}

Now, where to put your chart - in most cases a preferred way is to have a separate repo for all Ops CD files, including Helm chart - so this is where Helm chart would live. This is not to be mixed with CI files, such as Dockerfile - those should live alongside microservice repositories themselves.

3
  • Y, i'm leaning toward having my Helm charts in our separate Ops Git repo (we do this for ECS now). I'll explore this idea, thanks!
    – Chris F
    Commented Feb 17, 2021 at 21:36
  • 1
    All micro services in one chart? So you just have 1 release for all your services? I don't think that's great. Your helm rollback could affect everything, even only if one thing is broken. I know it sounds like it should be fine, but I have seen case where, for example, the CRD API changes behind the scenes, so the rollback fails, and you need to update the CRD API in your yaml in order to proceed. Having everything in 1 release can lead to a mess
    – TigerBear
    Commented Jun 21, 2023 at 9:08
  • Yes, single release for all services. In a distributed system you trade liveness for safety and you can never get 100% liveness guarantee - so no matter what you do you will find an example where liveness fails. Now, with a single release you always move from one well tested state to another - which is a strict requirement in many cases. Finally, for mission critical applications, test your rollback plan before the deployment - this will help avoid issues like the one you mentioned.
    – taleodor
    Commented Jun 21, 2023 at 9:31
3

I believe what you're asking is: "Is there any way to create just one helm chart that can be used for all microservices in my application?". If so, then you can just use the values.yaml file to store all the values for your templates. This is not considered good practice, considering your template file needs to hold the information for each of your microservice deployments (and thus will become really difficult to manage), but it is possible.

One example: say you have two microservices, and you need one Helm chart that will create the template for both microservices. Generally, you would create separate templates for each service under the templates folder, and deploy each Helm chart for each service individually, but instead you could create multiple deployments in one template yaml file, like

# For service 1
apiVersion: apps/v1
type: Deployment
etc., etc... (stick in all values.yaml file values here for service 1)
---
# For service 2
apiVersion: apps/v1
type: Deployment
etc., etc... (stick in all values.yaml file values here for service 2)

In your values.yaml file, you would then just place in the values for each of your services, like

# Service 1 Keys/Values
foo: value
# Service 2 Keys/Values
bar: otherValue

So to answer your question, you can package all your services into one individual Helm chart using the above method, and Kubernetes will run each service as their own ReplicaSet as expected. However, when you have many services to manage, it can be tricky to manage the template YAML files and the values.yaml files when you put in values for all your services in one file, and so it's most likely not a good practice to do this.

This is just my understanding of Helm so far, as I'm still learning Helm myself. As such, I'm not 100% sure if this can be done, so you might want to double check with another person that this answer is actually correct.

EDIT: To summarize: like I mentioned above, in theory, it is possible to use one Helm chart per service. But in practice, it will be extremely messy later on to manage the values.yaml and the templates in the templates folder. So the answer would be, "Yes in theory, but not recommended at all".

7
  • yes, this is exactly what I want. But what will be the directory structure to accomplish this? Each microservice is in it's own Git repo. So where will be the "generic" Helm charts be? Where will the microservice-specific values.yaml files be? How do I consolidate the two directory files? Remember, I want to be able to do a helm install ... from some diretory that has the complete chart.
    – Chris F
    Commented Feb 17, 2021 at 20:52
  • @ChrisF You would only need to put the regular Helm directory structure (Chart.yaml, values.yaml, and the templates folder) in one service. At that point, you would need to build your Docker images/push them to the registry first. Then, because K8s references only images stored in the registry, when you install your Helm chart using the files in just one microservice, K8s can pull the container image for all of your services and run it as expected. So when you run helm install from one service, it will work, as Kubernetes doesn't reference the directory, it references the container image. Commented Feb 17, 2021 at 21:07
  • I think this is a bad way to do it.
    – Chris F
    Commented Feb 17, 2021 at 21:10
  • 1
    Yes, like I said earlier this isn't a good way to go about it all, it makes more sense to just package each service using Helm separately, as this method is extremely messy. But, this shows it's possible. This is not the best way to do it though. Commented Feb 17, 2021 at 21:11
  • 1
    Then the correct answer is "No, you need a separate Helm chart per microservice."
    – Chris F
    Commented Feb 17, 2021 at 21:16
0

I have worked multiple jobs where I wrote just one helm chart as a base for all services. Within that, if written correctly, you can call any dependency you may need.

A common library chart is a reusable, and composable chart; designed to encapsulate common configurations, templates, and logic that can be shared across multiple Helm charts. The common library chart serves as building blocks or modules to compose all of your internal Kubernetes Manifests. Overall we utilize the common library chart to manage Kubernetes services as cattle vs sheep.

Below are some key points explaining the setup of the library chart.

Key Points: Reusability: Library charts promote code reuse by encapsulating commonly used configurations, templates, and logic. This reduces duplication and ensures consistency across multiple Helm charts.

Composition: Library charts can be combined with other charts to build more complex applications or services. This allows you to create modular and scalable Helm charts by composing smaller, reusable components.

Encapsulation: Library charts abstract away complex configurations and templates behind a simple interface, making it easier to manage and maintain Helm charts. Users can interact with library charts through well-defined parameters and values. Like in our README.md files fields.

Common Library Usage: To use the common library chart in the main application chart:

We declare the library chart as a dependency in the Chart.yaml file. Configure any parameters exposed by the library chart in our values.yaml file. Use the templates and resources provided by the common library chart in the templates/all.yaml file Drilling Down Example:

# Top level Chart.yaml
dependencies:
- name: common
  version: 1.0.0
  repository: file://lib/common

# Top level values.yaml files referring to common chart variables
configmap:
  config:
    enabled: true
    data:
      testing: "hay"

controller:
  enabled: true

image:
  repository: nginx
  pullPolicy: IfNotPresent
  # Overrides the image tag whose default is the chart appVersion.
  tag: "latest"
...

# Top level templates/all.yaml call the all() function inside common library chart.
{{- include "common.all" . }}

# Definition of the function all() inside lib/common/templates/_all.tpl
{{/*
Main entrypoint for the common library chart. It will render all underlying templates based on the provided values.
*/}}
{{- define "common.all" -}}
  {{- /* Merge the local chart values and the common chart defaults */ -}}
  {{- include "common.values.setup" . }}

  {{ include "common.configmap" . | nindent 0 }}

  {{- if .Values.controller.enabled }}
    {{- if eq .Values.controller.type "deployment" }}
      {{- include "common.deployment" . | nindent 0 }}
    {{ else if eq .Values.controller.type "daemonset" }}
      {{- include "common.daemonset" . | nindent 0 }}
    {{ else if eq .Values.controller.type "statefulset"  }}
      {{- include "common.statefulset" . | nindent 0 }}
    {{ else }}
      {{- fail (printf "Not a valid controller.type (%s)" .Values.controller.type) }}
    {{- end -}}
  {{- end -}}

  {{ include "common.cronjob" . | nindent 0 }}

  {{ include "common.classes.hpa" . | nindent 0 }}

  {{ include "common.ingress" .  | nindent 0 }}

  {{ include "common.job" . | nindent 0 }}

  {{ include "common.kongingress" . | nindent 0 }}

  {{ include "common.kongplugin" . | nindent 0 }}

  {{ include "common.poddisruptionbudget" .  | nindent 0 }}

  {{- include "common.pvc" . }}

  {{- if .Values.secret -}}
    {{ include "common.secret" .  | nindent 0 }}
  {{- end -}}

  {{ include "common.service" . | nindent 0 }}

  {{- if .Values.serviceAccount.create -}}
    {{- include "common.serviceAccount" . }}
  {{- end -}}

  {{ include "common.crb" . | nindent 0 }}

  {{ include "common.secrets" . | nindent 0 }}

{{- end -}}
1
  • Cool, I'll look at this. Thanks!
    – Chris F
    Commented Apr 23 at 15:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.