3

Using the eksctl tool, I created an EKS cluster with 5 nodes. My application's docker images are stored in ECR registries in the same region. I deployed my kubernetes cluster and everything has been happy for the past 6 weeks or so.

This morning, I came in and found 3 pods were in an ErrImagePull state. Using kubectl describe pod <pod name>, I found the error:

Failed to pull image "<ECR REGISTRY>/<IMAGE>": rpc error: code = Unknown desc = Error response from daemon: Get <ECR REGISTRY>/<IMAGE>: no basic auth credentials

My understanding of EKS and ECR is that I don't need a pull secret (and I haven't used one for any of the other running pods) so my guess is that some process or docker image on that node died but I can't find any docs on this.

Update: I forgot all about this question. When I created the original node group, I failed to include the --ssh-access flag which prevented me from getting onto the node and see if a kubernetes process had failed. I never found the actual solution; I simply added a taint to the problem node, created a new node, and went about my business. I'm still trying to find time to spin up a new node group with ssh access.

2
  • Did you find a solution to this?
    – Chris F
    Feb 16, 2021 at 1:21
  • Nope. Not related to this probe, we started running an internal docker registry with nexus instead.
    – Matthew
    Feb 17, 2021 at 23:26

1 Answer 1

1

Do your IAM roles that are attached to EC2 instances that are in EKS cluster have ECR iam policies? If not please update IAM roles Ref Link: https://docs.aws.amazon.com/AmazonECR/latest/userguide/ECR_on_EKS.html#:~:targetText=The%20Amazon%20EKS%20worker%20node,policy%20permissions%20for%20Amazon%20ECR.&targetText=When%20referencing%20an%20image%20from,tag%20naming%20for%20the%20image.

1
  • Yes, the IAM role has the correct permissions.
    – Matthew
    Nov 12, 2019 at 15:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.