How to exclude a node in kubernets using podAntiAffinity?

Question

I'm trying to exclude a set of nodes when I submit a job in kubernetes. The set of nodes do not (yet) have a label to differentiate them, so all I can exclude them on is hostname (for now).

All the examples on using podAntiAffinity use labelSelector which expects a label on the pod to differentiate it.

How can I limit my job from deploying to a specific set of nodes? The closest example I could scrape together from the docs is this non working example:

affinity:
  podAntiAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
    - labelSelector:               <--- this is wrong, there are no labels
      matchExpressions:
      - key: "kubernetes.io/hostname"
        operator: In
        values:
          - hostname1
          - hostname2

Answer 1

To exclude a set of nodes when submitting a job in kubernetes.

you should use Node affinity which is conceptually similar to nodeSelector –and will allow you to constrain which nodes your pod is eligible to be scheduled on, based on labels on the node and you should be able to use hostname.

On below example i run a job to 5 completions and all are started on node-01 of my cluster with node affinity defined.

Example : job.yaml

apiVersion: batch/v1
kind: Job
metadata:
  creationTimestamp: null
  labels:
    run: busybox
  name: busybox
spec:
  completions: 5
  template:
    metadata:
      creationTimestamp: null
      labels:
        run: busybox
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: kubernetes.io/hostname
                operator: In
                values:
                - k8s-node01
      containers:
      - image: busybox
        name: busybox
        resources: {}
      restartPolicy: OnFailure

Result :

ubuntu@k8s-master:~$ kubectl get all -o wide
NAME                READY   STATUS      RESTARTS   AGE     IP               NODE         NOMINATED NODE   READINESS GATES
pod/busybox-2gdp6   0/1     Completed   0          3m20s   192.168.85.204   k8s-node01   <none>           <none>
pod/busybox-l654d   0/1     Completed   0          3m55s   192.168.85.201   k8s-node01   <none>           <none>
pod/busybox-n7dbr   0/1     Completed   0          3m29s   192.168.85.203   k8s-node01   <none>           <none>
pod/busybox-pfc79   0/1     Completed   0          4m11s   192.168.85.200   k8s-node01   <none>           <none>
pod/busybox-r4xln   0/1     Completed   0          3m16s   192.168.85.205   k8s-node01   <none>           <none>

NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE   SELECTOR
service/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   31h   <none>

NAME                COMPLETIONS   DURATION   AGE     CONTAINERS   IMAGES    SELECTOR
job.batch/busybox   5/5           60s        4m11s   busybox      busybox   controller-uid=e12f136b-d828-4f7b-a49b-303499496a8a

I had two worker nodes on my cluster on job yaml i defined affinity to node01 only.

$ kubectl get nodes
NAME         STATUS   ROLES    AGE   VERSION
k8s-master   Ready    master   31h   v1.17.2
k8s-node01   Ready    <none>   31h   v1.17.2
k8s-node02   Ready    <none>   31h   v1.17.2