4

Say I have 2 nodes for a kubernetes cluster. Both has

2 CPU → 4 Total
2G RAM → 4 Total

I have 4 pods to shcedule-- no replicas yet, 1 major pod that'll use 80% of resources, other use 5% each. What resource limits should I specify for these pods?

CURENT USAGE STATS

# kubectl top pod --all-namespaces
NAMESPACE       NAME                        CPU(cores)   MEMORY(bytes)
default         pod1-64c455bcf8-mzxqc           1m          122Mi           
default         pod2-6f7b44478b-pn4tt           1m          33Mi            
default         pod3-867c956697-mf6zw           0m          48Mi            
default         pod4-844bd559dd-z96fh           0m          35Mi
1

The DevOps toolkit 2.3 book by Victor Farcic goes into some detail on this.

What You Should Understand

  • You as a developer specify whatever requests/limits you like initially.
  • These values have no baring on reality.
  • In order to get "real" values, you must monitor your application's real resource usage.
  • This is best done through a monitoring tool like prometheus, instana, datadog etc.
  • E.g. in instana, I can see that I have 512MB set up for a pod's requests, 2G for its limits, but the pod only has one container currently taking 57MB.
  • But I can't just use 60MB as a value now... I need to load test my application and see how it behaves under real/stressed circumstances.
  • Then you review these values and you can adjust the requests/limits accordingly.

Determine Usage Without a Monitoring Tool

You can use some kubectl commands to get these values as a one off (it's annoying at scale). Here is an example of a different app I have running.

Notice that requests/limits are set to 1/4-2 CPUs and 4G-8G memory. But the container is really using almost no CPU and 1.198G memory. If I load tested this and those values did not move up, I would need to change them. But I have load tested it and know that they would go up towards these realistic requests/limits, so it's okay.

$ kubectl describe pod -n gazelle gazelle-ws-0-0-1-6d67cffcf-lvlhk

...
    Limits:
      cpu:     2
      memory:  8Gi
    Requests:
      cpu:      250m
      memory:   4Gi

$ kubectl top pod gazelle-ws-0-0-1-6d67cffcf-lvlhk -n gazelle --containers

POD                                NAME         CPU(cores)   MEMORY(bytes)   
gazelle-ws-0-0-1-6d67cffcf-lvlhk   gazelle-ws   5m           1198Mi 

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.