Using AKS, the cluster has limits that are overcommitted due to kube-system
pods having excessively high CPU/memory limits.
kube-system
pods likecoredns
or have a combined CPU limit of 300+%- This is for a development environment. We have tried several different VM sizes like
Standard_B2ms
orStandard_D2_v3
which do not seem to affect thekube-system
pods limits/configurations. - We cannot manually reduce the limits for these
kube-system
pods because it is subsequently reverted by AKS. - Creating an additional node pool did not seem to solve the problem. Node taints were not respected.