1

In a stack with higher number of services it seems that Docker DNS does not work anymore which leads to effects that container's names are not known even to themselves.

While I have filed this bug, I wonder whether there is a way to debug this more extensively to "prove" the DNS error, or for example manually "fix" the DNS name.

Will be there some deeper error message?

Note: The ping from outside does not work; I just find cumbersome that is not possible to ping your own DNS name.

  • Did you login to several containers and check whether ping other-dockername works? When I check the docker-compose, I read cassandra-seed. So ping cassandra-seed from one of the docker containers should work. Does it? – 030 Aug 9 '17 at 14:58
  • The ping from outside does not work; I just find cumbersome that is not possible to ping your own DNS name. – Peter Aug 10 '17 at 19:12
  • What do you mean with outside, e.g. outside the POD, the k8s cluster? – 030 Dec 10 '17 at 10:30
  • It’s not too difficult to open an interactive bash shell inside a docket instance. Could ping via that and verify what the system’s actual state is: askubuntu.com/questions/505506/ddg#507009 – Moritz Jan 9 '18 at 18:24
1

There is a comprehensive document about k8s' DNS. According to this document one could validate whether the DNS is working by running:

busybox.yaml

apiVersion: v1
kind: Pod
metadata:
  name: busybox
  namespace: default
spec:
  containers:
  - image: busybox
    command:
      - sleep
      - "3600"
    imagePullPolicy: IfNotPresent
    name: busybox
  restartPolicy: Always

and deploy it by issuing:

kubectl create -f busybox.yaml

Once deployed, one could run:

kubectl get pods busybox

and validate whether the DNS is working:

kubectl exec -ti busybox -- nslookup kubernetes.default

There are additional validation steps that could be executed, including:

kubectl exec busybox cat /etc/resolv.conf

verify the DNS policy, check whether the DNS pod runs, checking erros in the DNS pod:

kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c kubedns
kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c dnsmasq
kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c sidecar

does the DNS service run?

kubectl get svc --namespace=kube-system

exposed DNS endpoints?

kubectl get ep kube-dns --namespace=kube-system

There are also multiple known issues regarding the k8s' DNS:

Linux’s libc is impossibly stuck (see this bug from 2005) with limits of just 3 DNS nameserver records and 6 DNS search records. Kubernetes needs to consume 1 nameserver record and 3 search records. This means that if a local installation already uses 3 nameservers or uses more than 3 searches, some of those settings will be lost. As a partial workaround, the node can run dnsmasq which will provide more nameserver entries, but not more search entries. You can also use kubelet’s --resolv-conf flag.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.