Is a ReplicaSet pulling a fresh image at every spin up?

Question

I inherited an app without a ton of handoff information, and there's a Kubernetes setup with ReplicaSets and KOPS (and based on Docker containers). I am pretty new to Kubernetes

Whenever one of the associated servers goes down, a cloned server is immediately spun up, which is working as desired.

However I am looking to make some persistent changes (uploading a some new files to the API, changing some environmental variables) to the setup, and my understanding is that I'll probably need to modify the images pulled to do this. Persistence is setup via PersistentVolumeLabel and openlayer

Is this correct?

I am trying to pull the images down locally so I can modify them, but getting an authentication required error - I have about a dozen private keys, so I assume the credentials are in one of them - is there a way to see how the ReplicaSet is pulling the images down?

Sorry for any confusions - I am very new to Kubernetes and this level of DevOps - I am used to much simpler and less scalable (so like a couple servers and a load balancer) type setups, not this type of DevOps.

Answer 1

my understanding is that I'll probably need to modify the images pulled to do this.

Yes and no. It depends on how your ReplicaSet - specifically the associated Deployment - is configured. You can see the corresponding YAML configuration by doing

kubectl -n yournamespace get deployment yourappdeployment -o yaml
# or 
kubectl -n yournamespace describe deployment yourappdeployment

Which will give you information about environment variables, images used, volumes, secrets, etc.

I am trying to pull the images down locally so I can modify them

Pulling an image down and modify it in-place won't be of much help, except if you run a container based on this image and use something like docker commit yo actually update the image. Even if you do that, the Kubernetes objects may override aspects defined in your image (such as environment variables)

is there a way to see how the ReplicaSet is pulling the images down?

Yes, first output the details of your ReplicaSet:

$ kubectl -n yournamespace get replicaset yourreplicaset -o yaml

kind: ReplicaSet
spec:
  ...
  template:
    ...
    spec:
      ...
      containers:
        ...
      imagePullSecrets:
      - name: some-secret-name

imagePullSecrets is the secret which should contains the credentials to pull your images. You can retrieve the data with:

$ kubectl -n yournamespace get secret some-secret-name -o yaml

kind: Secret
...
data:
  .dockerconfigjson: eyJh...VeryLongBase64String

This is Docker authentication data encoded in Base64. You must decode them:

$ echo "eyJh...VeryLongBase64String" | base64 -d

{"auths": {"registry.yourcompany.com": {"auth": "AnotherBase64EncodedString"}}}

Decode the auth value and you'll have your Docker registry login/password

$ echo "AnotherBase64EncodedString" | base64 -d

username:password

Answer 2

I am trying to pull the images down locally so I can modify them

Container images are immutable. The correct way to do this is to look for the Dockerfile (if you are using docker as the container engine) that corresponds to the image and make the relevant changes, rebuild the image and push it to the container registry.

The container registry will be evident to you in the Deployment manifest. You are looking for spec.containers.image attribute.

To answer your question specifically around "is the ReplicaSet pulling a fresh image", this depends on the imagePullPolicy setup in the manifest. You can see the definition below:

FIELD:    imagePullPolicy <string>

DESCRIPTION:
     Image pull policy. One of Always, Never, IfNotPresent. Defaults to Always
     if :latest tag is specified, or IfNotPresent otherwise. Cannot be updated.
     More info:
     https://kubernetes.io/docs/concepts/containers/images#updating-images