2

I have a private docker registry in k8 in the default namespace with tls at https://docker-registry.default:5000. By doing local port forwarding to it(at port 5000) and adding docker-registry.default to my /etc/hosts file, I have been able to pull and push images to it.

However, I'm using a serverless framework(Nuclio) which manages the deploy for me. The error I get is:

Normal   Scheduled  7m52s                   default-scheduler                                       Successfully assigned nuclio/helloworld1-96876fcf9-7c87w to gke-your-first-cluster-1-pool-1-fe915942-gnkx
  Normal   Pulling    6m14s (x4 over 7m51s)   kubelet, gke-your-first-cluster-1-pool-1-fe915942-gnkx  Pulling image "docker-registry.default:5000/docker/nuclio/processor-helloworld1:latest"
  Warning  Failed     6m14s (x4 over 7m51s)   kubelet, gke-your-first-cluster-1-pool-1-fe915942-gnkx  Failed to pull image "docker-registry.default:5000/docker/nuclio/processor-helloworld1:latest": rpc error: code = Unknown desc = Error response from daemon: Get https://docker-registry.default:5000/v2/: dial tcp: lookup docker-registry.default on 169.254.169.254:53: no such host
  Warning  Failed     6m14s (x4 over 7m51s)   kubelet, gke-your-first-cluster-1-pool-1-fe915942-gnkx  Error: ErrImagePull
  Normal   BackOff    6m2s (x6 over 7m51s)    kubelet, gke-your-first-cluster-1-pool-1-fe915942-gnkx  Back-off pulling image "docker-registry.default:5000/docker/nuclio/processor-helloworld1:latest"
  Warning  Failed     2m38s (x21 over 7m51s)  kubelet, gke-your-first-cluster-1-pool-1-fe915942-gnkx  Error: ImagePullBackOff

It gives a no such host error.

A Nuclio developer has confirmed that it's not a nuclio issue since I cannot even do a docker login from the Nuclio Admin pod(not the failed pod above) to this internal docker-registry.default though I can do a docker login to the Docker hub and a wget to docker-registry.default successfully. The Nuclio Admin pod is Alpine Linux if it helps.

1

It seems your private DNS is misconfigured. Check your host or make sure that docker-registry.default is the a valid hostname.

You can run a temporary Ubuntu pod to check if that hostname is resolving correctly from your cluster, e.g.

$ kubectl run ubuntu -it --image ubuntu --rm=true -- bash
root@shell:/# apt update && DEBIAN_FRONTEND=noninteractive apt --yes install dnsutils
root@shell:/# dig docker-registry.default
docker-registry.default.    IN  A D0.y0u.h4ve.1P?

Additional suggestions:

  • If you're using a cloud provider (such as Azure), make sure your Private DNS zone is linked with your cluster's virtual network. Read: Connect privately to an Azure container registry using Azure Private Link
  • If your container registry endpoint isn't correct, edit the deployment manifest and fix the image path.
  • Test pulling the image from another VM within the same private network.
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.