4

I have a Kubernetes Cluster on GCE (Google Cloud Engine) (Also called GKE - Google Kubernetes Engine).

The Cluster is configured to have 4cpu per node. The pods I am using request 2cpu. I expect to be able to have 2 pods per node. But instead each time I launch a new pod, it also provisions a new node (am only getting 1 pod per node).

What is happening, How can I more efficiently use my resources? (I don't want to pay for nodes that are under-utilized)

Node:

Machine type
n1-highcpu-4 (4 vCPUs, 3.6 GB memory)

Pod (Yaml):

resources:
  limits:
    cpu: "2"
    memory: 1000Mi
  requests:
    cpu: "2"
    memory: 1000Mi

EDIT:

As mentioned below in the comments; it seems there are some 'default' pods that run on the node which utilize some small amount of CPU. For me there were two service pods taking up 100mcpu each. Additionally it seems there is a small overhead regardless of those small pods when viewing my node resources I see:

Resource type   Capacity   Allocatable   
CPU             4 CPU      3.92 CPU

This means that each node in my cluster will only have 3.72 CPU that I can distribute to my pods. (3.92 allocatable, - 100m CPU per service pod (x2) = 3.72)

Knowing this, I can understand why my desired deployment will not work.

However, now this raises new questions; How can I most efficiently allocate pods that need 2cpu. Since GCE only lets me provision nodes with even numbers of cpu (2,4,6,etc) I will always have almost 2 wasted CPU. Is there a better solution?

EDIT2:

Settings my pods to use 1.8 CPU allowed me to deploy 2 pods per node. (and for now is good enough) It would be desirable to be able to anticipate the overhead on nodes. For example, in my use-case above, allocating 4CPU per node, I should be able to know ahead of time that after the service pods and allocatable CPU are considered I will only be able to use 3.72 for my own pods. But if I now want to provision a new Node with 8CPU I will need to check these numbers all over again.

Improve this question
6
  • 1
    IIRC you have other pods runnings on each node, at least one for K8S itself, so obvisouly you can't use 4 cpu with your own 2 pods as there's already 0.1 or 0.2 used by other pods than yours and the scheduler knowing that raise a new node to host the new pod of 2cpu, try with 1.8 to see if it change the behavior.
    – Tensibai
    Jun 13, 2018 at 14:20
  • nodes can only be defined on GCE using even numbers (2,4,6,etc) meaning that if i want to have a pod that uses 2cpu I will always be under-utilizing my node by almost 2 cpus? this seems like a serious waste of resource. Surely there must be a solution.
    – Inbar Rose
    Jun 13, 2018 at 14:38
  • A 4cpu node can handle at least three 1 cpu pod, a pod needing absolutely 2 cpu sounds like someting which has nothing to do in a containers where we usually play with less than 1 cpu reservation per pod.
    – Tensibai
    Jun 13, 2018 at 14:45
  • 1
    Try with cpu: "1.7" in your requests blocks and see if it really is a problem for your pods...
    – Tensibai
    Jun 13, 2018 at 14:52
  • My use-case requires strong CPU. I can probably allocate 1.8 and it will be fine. But a 2cpu pod should not be such a strange use-case, nor a hard to accommodate deployment. And it's frustrating that there is so much overhead/constraints.
    – Inbar Rose
    Jun 13, 2018 at 14:52

1 Answer 1

Reset to default
1

It seems like you found your answer but let me add a little color since we went through the same thing.

Your yaml for the pods can have both a request and limit for both CPU and memory. Since you are discussing CPU here I'll stick with that. What you can do in order to utilize your resources efficiently is set your resource request to the minimum requirement to get scheduled. Say 1.8. But also set the limit to a higher limit, say 2 or more. What this will do is make your pod be schedulable on the machine and if CPU is not completely consumed, your pod will utilize the available CPU up to 2 or the limit you set. This is the way kubernetes can make good use of the resources by "sharing" CPU to the application that needs it.

See here for requests and limits: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/

Improve this answer
2
  • What would happen if say, I had a node with 4 CPU (meaning 3.72 available) and I scheduled a pod with request 1.8 and limit 2. The pod would consume 2 CPU since nothing else wants it, but then if i wanted to schedule another pod with request 1.8 limit 2 it would not have enough CPU? or would it reduce the CPU of the pod that is already there?
    – Inbar Rose
    Jun 27, 2018 at 7:24
  • Kubernetes will schedule the pod based on an algorithm of what's available. So if it is consuming the 2 (or more) it won't be scheduled. But if you have burst loads then it will schedule when the burst is down. You request basically your minimum average. I would say if your load isn't burstable or your worried about not having the .2 cores per process I would upgrade to the 6 to leave room for pods going up and down. Then your overhead (wasted .28 cpu's) is only that one. Unless you are worried about HA, rolling updates etc then one vm isn't going to be good. Hope that helps
    – Jeff Garrett
    Jun 28, 2018 at 21:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.