1

I can connect to the Kubernetes API from a container running inside the cluster with an OIDC token like so:

kubectl —token $token get pod

I would now like to prepare a kubeconfig such that other programs that don’t support --token can connect as well. I am trying this in the specific context of docker buildx create --driver kubernetes ...; docker buildx build . where the 2nd part needs to create a deployment. Here's how I have proceeded:

tmp=$(mktemp -p $PWD)
kubectl config --kubeconfig $tmp set-cluster cluster \
  --server https://$KUBERNETES_SERVICE_HOST:$KUBERNETES_SERVICE_PORT \
  --certificate-authority /var/run/secrets/kubernetes.io/serviceaccount/ca.crt 
kubectl config --kubeconfig $tmp set-credentials user --auth-provider oidc \
  --auth-provider-arg idp-issuer-url=... \
  --auth-provider-arg client-id=... \
  --auth-provider-arg id-token=$token
kubectl config --kubeconfig $tmp set-context context --cluster cluster --user user
kubectl config --kubeconfig $tmp use-context context
export KUBECONFIG=$tmp
kubectl get pod

kubectl get pod now fails with "Unable to connect to the server: No valid id-token, and cannot refresh without refresh-token". Why does it deem $token invalid when it has worked just before with kubectl get pod --token (and still does)? How can create a kubeconfig "envelope" around $token such that kubectl get pod will accept it and won't fail? (I've not specified refresh-token because it shouldn't be needed in the case of a valid token and because I don't have it readily available inside the container.)

1

I have come to the (preliminary) conclusion that this cannot be done.

First, I had set the wrong idp-issuer-url. I had selected the IDP that normally serves during OIDC authentication to the Kubernetes cluster. But $token (or $CI_JOB_JWT in GitLab CI/CD parlance) was not generated there but by GitLab instead. Inspection e.g. at jwt.io indicates "iss": "kubernetes/serviceaccount", so --auth-provider-arg idp-issuer-url=kubernetes/serviceaccount should be a better choice.

But this inspection also reveals that the JWT does carry a claim aud, hence no client-id. kubectl in particular (and perhaps OIDC in general) apparently insist on a non-empty client-id (otherwise: "error: Must provide client-id"). So I guess what happens is that kubectl checks the issuer (signer) and client-id for $token (claims iss and aud in its JWT payload) before it will use it. The check for client-id must fail, at which point kubectl tries a token refresh and that fails again.

If that diagnosis is correct this cannot be done because kubectl insists on a client-id whereas GitLab does not provide one in $token. I guess the reason for the discrepancy is that GitLab's is a general JWT token, whereas kubectl (more specifically, its OIDC autentication provider) expects a OIDC id tokens in particular (unless it is invoked with --token, which apparently bypasses kubeconfig and hence the OIDC authentication provider).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.