I am a beginner in kubernetes. I have a cluster running with 4 nodes. All the nodes are currently running fine. They are not tainted, or drained. Every time I deploy a deployment it will only deploy it across 2 nodes instead of 4 nodes. I am trying to look for kube-scheduler logs, but because I am using Oracle Cloud OKE service (kubernetes service for Oracle cloud). They don't make kube-scheduler logs available to customers. What tests or what verifications do I need to do?
If you want to place a pod on each node you can use DaemonSet. DaemonSets are configured to distribute the application to each node. For detailed information;
You can find out if DaemonSet is what you need by looking specifically at this title;
The scheduler placing your pods on one or two nodes, until they're out of resources, could be a "normal" behavior depending on your cluster scheduler policy.
You could try to use an antiAffinity rule, or a topologySpreadConstraints, placing your pods on all 4 nodes.
The following sample, from Kubernetes docs, would use topology key 'kubernetes.io/hostname', which should spread your pods on different nodes:
apiVersion: apps/v1 kind: Deployment metadata: name: web-server spec: selector: matchLabels: app: web-store replicas: 3 template: metadata: labels: app: web-store spec: affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: app operator: In values: - web-store topologyKey: kubernetes.io/hostname
requiredDuringSchedulingIgnoredDuringExecution would prevent the Kubernetes scheduler from ever starting a replicas on a node that already has a Pod that belongs to your deployment.
We could do something similar with a
apiVersion: apps/v1 kind: Deployment metadata: name: web-server spec: selector: matchLabels: app: web-store replicas: 3 template: metadata: labels: app: web-store spec: topologySpreadConstraints: - maxSkew: 1 topologyKey: kubernetes.io/hostname whenUnsatisfiable: DoNotSchedule labelSelector: matchLabels: app: web-store
1The OP didn't include a copy of their manifest, but one can likely safely assume that given they are talking about "deployments" that they are deploying a ReplicaSet (the default for deployment.apps) not a DaemonSet, which means that there would only be spec.replicas pods deployed, regardless of the node count. Only a DaemonSet would deploy as many replicas as there are nodes, subject to node affinity/topology constraints.– KyanarJul 24, 2022 at 8:02
1There are no mentions to replicas in initial question. Question is: why is it spreading on two nodes. There is no mention of a requirement to run "1 replica per node" either. Very much sounds like OP wants to know why, adding replicas, they're not spreading.– SYNJul 24, 2022 at 12:58