0

I am trying to resolve a recurring Prometheus alert I'm getting that states that my Kubernetes cluster etcd database is fragmented and should be defragmented. I have never used the etcdctl command before, which is apparently installed automatically in the container in which etcd is running.

I have found a post that gives a very brief description of what to do, but it does not work for me. I have also been looking at documentation on the etcd.io site, but everything there seems to assume the user is in the pod, running the etcdctl command directly.

The command I have tried to use is:

kubectl exec $(kubectl get pods --selector=component=etcd -A -o name | head -n 1) -n kube-system -- etcdctl defrag --cluster --cacert /etc/kubernetes/pki/etcd/ca.crt --key /etc/kubernetes/pki/etcd/apiserver.key --cert /etc/kubernetes/pki/etcd/apiserver.crt

I am not 100% sure if I'm specifying the correct file paths for the --key and --cert options. The first link just uses server.key and server.crt, which it complains don't exist for my set-up. I did find the files apiserver.key and apiserver.crt, but maybe those are not the files to use. If they are not, I don't know what should be used.

I'm running this from Git Bash. It errors out with a message that seems to make no sense. It's almost as though it's trying to use the local filesystem (which is a PC running Windows 11).

The error message:

Error: open C:/Program Files/Git/etc/kubernetes/pki/etcd/apiserver.crt: no such file or directory
command terminated with exit code 128

Are these directions correct? Does anyone know why I might be getting the strange error?

UPADTE:

As suggested by @Botje, I ran the following (note the extra slash at the start of each file path):

$ kubectl exec $(kubectl get pods --selector=component=etcd -A -o name | head -n 1) -n kube-system -- etcdctl defrag --cluster --cacert //etc/kubernetes/pki/etcd/ca.crt --key //etc/kubernetes/pki/etcd/apiserver.key --cert //etc/kubernetes/pki/etcd/apiserver.crt

Unfortunately this did not work either:

Error: open //etc/kubernetes/pki/etcd/apiserver.crt: no such file or directory
command terminated with exit code 128

2 Answers 2

1

You have to login to that host where etcd pods is running (usually Mater node) and execute :

etcdctl defrag

OR

ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379  --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/server.crt --key=/etc/kubernetes/pki/etcd/server.key defrag

prerequisite it to install etcdctl (by installing etcd to that host)

1

You need to run the command on one of the Kubernetes nodes where the etcd pods are running.

I'm running this from Git Bash. It errors out with a message that seems to make no sense. It's almost as though it's trying to use the local filesystem (which is a PC running Windows 11).

That's because this is exactly what you are doing, and your Windows 11 install doesn't have the certificate files.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.