How to replicate a mongodb pod with its persistent storage

Question

I made these settings :

    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      labels:
        name: mongo-claim0
      name: mongo-claim0
      namespace: my-app
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 100Mi
    status: {}

---

apiVersion: v1
kind: ReplicationController
metadata:
  labels:
    name: mongo
  name: mongo-controller
  namespace: my-app
spec:
  replicas: 1
  template:
    metadata:
      labels:
        name: mongo
    spec:
      containers:
      - image: mongo
        name: mongo
        ports:
        - name: mongo
          containerPort: 27017
        volumeMounts:
        - mountPath: /data/db
          name: mongo-claim0
      restartPolicy: Always
      volumes:
      - name: mongo-claim0
        persistentVolumeClaim:
          claimName: mongo-claim0

---

apiVersion: v1
kind: Service
metadata:
  name: mongo
  namespace: my-app
  labels:
    name: mongo
spec:
  ports:
  - port: 27017
    targetPort: 27017
  selector:
    name: mongo

When I try to scale this pod, Minikube UI show :

mongo-controller-xr21r -> Waiting: CrashLoopBackOff Back-off restarting failed container Error syncing pod

And I got an error on the new pod :

exception in initAndListen: 98 Unable to lock file: /data/db/mongod.lock Resource temporarily unavailable

Could you help me to be able to scale a pod with a persistent storage ?

Answer 1

https://www.mongodb.com/blog/post/running-mongodb-as-a-microservice-with-docker-and-kubernetes

MongoDB database nodes are stateful. In the event that a container fails, and is rescheduled, it's undesirable for the data to be lost (it could be recovered from other nodes in the replica set, but that takes time). To solve this, features such as the Volume abstraction in Kubernetes can be used to map what would otherwise be an ephemeral MongoDB data directory in the container to a persistent location where the data survives container failure and rescheduling.

Like @Tensibai indicated the issue is related to the replication. According to this blog it could be solved as follows.

 apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: mongo
spec:
  serviceName: "mongo"
  replicas: 3
  template:
    metadata:
      labels:
        role: mongo
        environment: test
    spec:
      terminationGracePeriodSeconds: 10
      containers:
        - name: mongo
          image: mongo
          command:
            - mongod
            - "--replSet"
            - rs0
            - "--smallfiles"
            - "--noprealloc"
          ports:
            - containerPort: 27017
          volumeMounts:
            - name: mongo-persistent-storage
              mountPath: /data/db
        - name: mongo-sidecar
          image: cvallance/mongo-k8s-sidecar
          env:
            - name: MONGO_SIDECAR_POD_LABELS
              value: "role=mongo,environment=test"
  volumeClaimTemplates:
  - metadata:
      name: mongo-persistent-storage
      annotations:
        volume.beta.kubernetes.io/storage-class: "fast"
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 100Gi