Entering to docker container of my elasticsearch google kubernetes pod - CONTAINER ID is changing

Question

I have a kubernetes cluster on GCP, in which I have a elasticsearch service. I have a elasticsearch-0 pod which is running a docker container of elasticsearch.

~ ❯❯❯ kubectl describe pod elasticsearch-0
Name:           elasticsearch-0
Namespace:      default
Node:           gke-jaeger-persistent-st-default-pool-xxxxx-phvx/10.166.0.4
Start Time:     Mon, 07 Jan 2019 14:21:19 +0100
Labels:         app=jaeger-elasticsearch
                controller-revision-hash=elasticsearch-8684f69799
                jaeger-infra=elasticsearch-replica
                statefulset.kubernetes.io/pod-name=elasticsearch-0
Annotations:    kubernetes.io/limit-ranger: LimitRanger plugin set: cpu request for container elasticsearch
Status:         Running
IP:             10.36.2.27
Controlled By:  StatefulSet/elasticsearch
Containers:
  elasticsearch:
    Container ID:  docker://5212b8c223401355bd29a5639caea5097074a8a8101ceb10300f76465e4a6536
    Image:         docker.elastic.co/elasticsearch/elasticsearch:5.6.0
    Image ID:      docker-pullable://docker.elastic.co/elasticsearch/elasticsearch@sha256:f95e7d4256197a9bb866b166d9ad37963dc7c5764d6ae6400e551f4987a659d7
    Port:          <none>
    Host Port:     <none>
    Command:

And I need access to the node which manage my pod in order to get the container ID of my elasticsearch container.

I get the name of my instances with gcloud compute instances list and enter via gcloud compute ssh <node-name> --zone=<zone-name> command and I've enter to the elasticsearch docker container with root privileges docker exec -it -u root containerID bash (following these steps)

I need perform this because I need add a new mountPath named /jaeger-data in order to associate it to a persistent volume claim that I have prepared.

I enter to my docker container and create the mountPath

[root@elasticsearch-0 elasticsearch]# mkdir /jaeger-data
[root@elasticsearch-0 elasticsearch]#

But I can see that I am disconnecting continously from my docker container because the containerID is changed of a continous way. Each time that I enter to my node to get my containerID with docker ps -a command, this containerID is different and when enter to the new container making use of the new ID, my /jaeger-data mountPath is not found, this mean that there isn't, may be because I am entering to a new container ...

What is the reason to my containerID change all time? Is this related with Kubernetes node that contain my pod which have my elasticsearch docker container?

Is related with the Service situation?

Kubernetes Pods are mortal. They are born and when they die, they are not resurrected. ReplicaSets in particular create and destroy Pods dynamically (e.g. when scaling out or in). While each Pod gets its own IP address, even those IP addresses cannot be relied upon to be stable over time. This leads to a problem: if some set of Pods (let’s call them backends) provides functionality to other Pods (let’s call them frontends) inside the Kubernetes cluster, how do those frontends find out and keep track of which backends are in that set?

Enter Services.

How to can I address this situation?

I need create my mountPath in order to associate it to my elasticsearch-0 pod configuration

Answer 1

If I understand you question clearly, you're trying to mount /jaeger-data inside the elastic-search container or pod. To achieve this, what you can do is put this mount details inside the statefulset yaml of the Kubernetes pod for elastic-search. Using the volumeMounts inside the statefulset, you will not need to ssh or exec inside the container or bother about the change or container ID for each pod restart/recreation. You can use the below example as a reference for your PVC and volumeMounts, just edit your statefulset using kubectl edit sts elasticsearch-0 -n default, this will open an editor and you can replace you PVC details in the below yaml and append and save and exit will recreate your pod and attach the mountpoint:

          volumeMounts:
            - mountPath: /es-storage
              name: jaeger-data
  volumeClaimTemplates:
  - metadata:
      name: es-storage
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 100Gi

• Container ID will change with each recreation of containers;
• If in case you need to exec into the container, the better way is to use kubectl exec -ti <pod-name> -n <namespace> bash to exec into the container from any of the master node(similar command as docker exec, just replace pod-name and namespace) or from your local laptop if you have kubeconfig configured.
• Using Service, you can expose this application on a particular port, Kubernetes creates an abstraction and itself takes care of the change of pod IP