I have a Kubernetes Cluster on GCE (Google Cloud Engine) (Also called GKE - Google Kubernetes Engine).
The Cluster is configured to have 4cpu per node. The pods I am using request 2cpu. I expect to be able to have 2 pods per node. But instead each time I launch a new pod, it also provisions a new node (am only getting 1 pod per node).
What is happening, How can I more efficiently use my resources? (I don't want to pay for nodes that are under-utilized)
n1-highcpu-4 (4 vCPUs, 3.6 GB memory)
resources: limits: cpu: "2" memory: 1000Mi requests: cpu: "2" memory: 1000Mi
As mentioned below in the comments; it seems there are some 'default' pods that run on the node which utilize some small amount of CPU. For me there were two service pods taking up 100mcpu each. Additionally it seems there is a small overhead regardless of those small pods when viewing my node resources I see:
Resource type Capacity Allocatable CPU 4 CPU 3.92 CPU
This means that each node in my cluster will only have 3.72 CPU that I can distribute to my pods. (3.92 allocatable, - 100m CPU per service pod (x2) = 3.72)
Knowing this, I can understand why my desired deployment will not work.
However, now this raises new questions; How can I most efficiently allocate pods that need 2cpu. Since GCE only lets me provision nodes with even numbers of cpu (2,4,6,etc) I will always have almost 2 wasted CPU. Is there a better solution?
Settings my pods to use 1.8 CPU allowed me to deploy 2 pods per node. (and for now is good enough) It would be desirable to be able to anticipate the overhead on nodes. For example, in my use-case above, allocating 4CPU per node, I should be able to know ahead of time that after the service pods and allocatable CPU are considered I will only be able to use 3.72 for my own pods. But if I now want to provision a new Node with 8CPU I will need to check these numbers all over again.