Why are unnamed volumes created when `docker-compose up` is run and do these double after issuing `docker-compose down && docker-compose up`?

Question

When starting a docker compose application, it creates named volume the first time around and then additional volumes each time it is started. Is this the normal behavior in that it creates incremental volumes or is my docker-compose file broken? In addition, after running docker volume prune, it removed two of the unnamed volumes

This is the output after starting it twice:

DRIVER              VOLUME NAME
local               2b33c81e6e955ff36061f4120b7181c7efc7aebded2f87eaa7290027f1e7a725
local               74301eadc75018d6f7da76aec44bcf743e03261492cfcc33211941684a570030
local               c42b1e2c17342c52b4b9e90f4d4b7fa24ccb30de5479e65cf4366e2373dfb0bf
local               dcfa63dbf99362fc92fe3b981f50abebfe1bba03063d4dd9dd790f4c058817cf
local               dockerinfluxdbgrafana_grafana-data
local               dockerinfluxdbgrafana_influxdb-data

And for reference my docker-compose file

version: "3"
services:
  influxdb:
    image: influxdb:latest
    container_name: influxdb
    ports:
      - "8086:8086"
    env_file:
      - 'env.influxdb'
    networks:
      - backend
    volumes:
      - influxdb-data:/var/lib/influxdb

  grafana:
    image: grafana/grafana:latest
    container_name: grafana
    ports:
      - "3000:3000"
    env_file:
      - 'env.grafana'
      - 'secrets.grafana'
    networks:
      - backend
    volumes:
      - grafana-data:/var/lib/grafana

networks:
  backend:

volumes:
  influxdb-data:
  grafana-data:

Steps to reproduce

docker-compose up -d && docker-compose down && docker-compose up -d && docker-compose down && docker-compose up -d && docker-compose down

Answer 1

Research indicated that the anonymous volumes were created by influxdb and grafana/grafana.

Anonymous volumes are not given an explicit name when they are first mounted into a container, so Docker gives them a random name that is guaranteed to be unique within a given Docker host. Besides the name, named and anonymous volumes behave in the same ways.

Results

version: "3"
services:
  influxdb:
    image: influxdb:latest
    container_name: influxdb
    ports:
      - "8086:8086"
    #env_file:
    #  - 'env.influxdb'
    networks:
      - backend
    volumes:
      - influxdb-data:/var/lib/influxdb

  grafana:
    image: grafana/grafana:latest
    container_name: grafana
    ports:
      - "3000:3000"
    #env_file:
    #  - 'env.grafana'
    #  - 'secrets.grafana'
    networks:
      - backend
    volumes:
      - grafana-data:/var/lib/grafana

networks:
  backend:

volumes:
  influxdb-data:
  grafana-data:

results in:

DRIVER              VOLUME NAME
local               604a07040367512b09c618c6dcc71a7f55390c9c23de6ab08be7466414ed62da
local               7f4b630073b31b6e772d3edef6da81b48643525edfc34281ea13fbd6b86ec270
local               devopsstackexchange_grafana-data
local               devopsstackexchange_influxdb-data

and everytime when docker-compose down and subsequenlty up is run the number of anonymous volumes doubles.

What will happen if nginx will be pulled instead of influxdb and grafana?

DRIVER              VOLUME NAME
local               devopsstackexchange_grafana-data
local               devopsstackexchange_influxdb-data

and after docker-compose down && docker-compose up -d?

DRIVER              VOLUME NAME
local               devopsstackexchange_grafana-data
local               devopsstackexchange_influxdb-data

It looks like that certain images created the additional anonymous volumes. Let's replace influxdb with nginx and use grafana.

DRIVER              VOLUME NAME
local               15b80416ab06abb629d9f634a0feff08f7c560f31d614b9b430855c16cdb75c7
local               205a6f19cbf992c95b2e3be9f2fb1ca9ecec35fce550d0b7a4b9f32b0ef163b1
local               474108f5b7b14fba92a3e5a980f3bf851388b2ee25d7417df5c42d9f176e084b
local               5830a31a470ec8a42ddae7a37bb50487f3f36360318b2f9f5301b338507782b4
local               9f00868a2fec0cfc0d34dc12d0879d39487a13128863722f400ad4c47df2d340
local               devopsstackexchange_grafana-data
local               devopsstackexchange_influxdb-data
local               f47b1b7bbec8e50b32a7c39704c7c218165b284298d852313fa24bc7cbe6acc5

Everytime docker compose was run again, three anonymous volumes were created by the grafana/grafana docker image. Let's replace influxdb with nginx and revert the grafana to nginx:

DRIVER              VOLUME NAME
local               devopsstackexchange_grafana-data
local               devopsstackexchange_influxdb-data

and it remains two if restarted.

It looks like that grafana is causing the issue.

Why are three new anonymous volumes created everytime grafana/grafana is restarted?

The grafana/grafana dockerfile indicates that three anonymous volumes will be created:

VOLUME ["/var/lib/grafana", "/var/log/grafana", "/etc/grafana"]

https://docs.docker.com/engine/reference/builder/#volume

FROM ubuntu
RUN mkdir /myvol
RUN echo "hello world" > /myvol/greeting
VOLUME /myvol

This Dockerfile results in an image that causes docker run, to create a new mount point at /myvol and copy the greeting file into the newly created volume.

docker volume inspect <volume name, e.g. 34cfafd4603dbc7e71a83e2520f978c8307b084143b3192de65a1995dc1d2f86> returned grafana data when the path that was returned was checked of two out of three anonymous volumes:

sudo ls /var/lib/docker/volumes/a0ecd00df8fc68ef36e777c7bf9ec5a496ee30e313b86889487501a53fa2e28e/_data
grafana.ini  ldap.toml

and

sudo ls /var/lib/docker/volumes/34cfafd4603dbc7e71a83e2520f978c8307b084143b3192de65a1995dc1d2f86/_data
grafana.db  plugins

Answer 2

Check the volumes defined in the Dockerfile with the VOLUME instruction.

If you don't want volume with random names to be automatically created, you can provide a host path to be bind-mounted to the volume, check the -v or --mount options, https://docs.docker.com/engine/admin/volumes/volumes/#choose-the--v-or-mount-flag

For example:

-v /srv/lib/grafana:/var/lib/grafana

Answer 3

If anyone else is going down this rabbit hole. Make sure you didn't accidentally define your volume with an = in your Dockerfile

Wrong:

VOLUME = ["/logs", "/data"]

Correct:

VOLUME ["/logs", "/data"]

It took me a long time to dig into why the anonymous volumes were being added by docker-compose