I have set up a private Docker registry on my Kubernetes cluster (k3s) running on a Raspberry Pi 4. I have secured the registry with a Let's Encrypt certificate, and I can confirm that the certificate is valid when accessing the registry from a web browser. However, when trying to deploy an application in Kubernetes that uses an image from this private registry, I am encountering an error related to the certificate.
Failed to pull image "192.168.42.50:5000/dev-backend:latest": rpc error: code = Unknown desc = failed to pull and unpack image "192.168.42.50:5000/dev-backend:latest": failed to resolve reference "192.168.42.50:5000/dev-backend:latest": failed to do request: Head "https://192.168.42.50:5000/v2/dev-backend/manifests/latest": x509: certificate signed by unknown authority
I have created a Certificate resource in the "docker" namespace and successfully obtained a certificate from Let's Encrypt. The certificate request and its corresponding certificate can be seen with the following commands:
root@atlas00:~# kubectl get certificaterequests -n docker NAME APPROVED DENIED READY ISSUER REQUESTOR AGE docker-registry-tls-hhm6j True True letsencrypt-prod system:serviceaccount:cert-manager:cert-manager 7h17m root@atlas00:~# kubectl get certificates -n docker NAME READY SECRET AGE docker-registry-tls True docker-registry-tls 7h17m
The registry is using the Let's Encrypt certificate correctly, and I have even added the Let's Encrypt root certificate (ISRG Root X1) to the trusted certificate store for the container runtime (Docker) on each node of my cluster.
Here is my deployment YAML file:
apiVersion: apps/v1 kind: Deployment metadata: name: dev-backend namespace: dev spec: replicas: 1 selector: matchLabels: app: dev-backend template: metadata: labels: app: dev-backend spec: containers: - name: dev-backend image: 192.168.42.50:5000/dev-backend:latest ports: - containerPort: 80 volumeMounts: - name: dev-backend-storage mountPath: /app/data volumes: - name: dev-backend-storage persistentVolumeClaim: claimName: dev-backend-pvc imagePullSecrets: - name: docker-registry-config
I have created a Kubernetes secret (docker-registry-config) containing the necessary credentials for accessing the private registry.
Despite these configurations, Kubernetes still reports the certificate as signed by an unknown authority. How can I ensure that Kubernetes trusts the Let's Encrypt certificate and successfully pulls the image from my private registry? Or is there a certificate authority that Kubernetes trusts out of the box? I'd like to avoid using an 'insecure-registry' approach. Thanks.