My CD pipeline:

  1. Get docker image (created and pushed during CI stage)
  2. Copy docker-compose .yml template to target server
  3. Run the app

Let's go straight to the 3rd point. I can run the app using docker-compose up e.g., but I have no idea how to handle configuration. Target server is a Ubuntu Server created by Chief on AWS, I have an access to all AWS services like Parameter Store etc.

So here are my ideas about managing configurations:

  1. save secrets to configuration file before running the app and attach this file to specific location in docker-compose template (I don't like this solution, I can handle this by some bash script that inject secrets to file)
  2. set environment variables before running the app (I talked with some devops and basically they like this solution)
  3. get secrets from AWS Parameter Store (unfortunately I can't have an access by role management, only by key which I need to set on server by env or in config file)

How managing configuration looks in real case scenarios? I'm worried about case of server crash or something similar. 1st solution will survive this, but in case of 2nd and 3rd ones the environment variables will be lost after crash/reboot. What is the best solution here?


If you start a container with an environment variable, that variable is part of the container definition. As a pro, when the server is rebooted, the same variable will be there when the container restarts. But as a con, anyone with access to the docker API can view that secret in clear text, and some apps log their environment when debugging.

With swarm mode (Kubernetes has their own similar solution), you can define your secret on the manager and it will be given to nodes running the container on a need to know basis. The secret is accessible inside the container as a file, but that file is only stored in memory on the worker node, and you can use an entrypoint script in your container to load that into the environment if that's how the application expects it. The swarm mode secrets and configs have the advantage of being portable as the container is rescheduled on other nodes in the event of a worker crash. Migrating from a docker-compose install to a single node swarm cluster and deploying a stack is a very simple task.

There are also external solutions for storing configs and secrets in key/value stores like CoreOS's etcd and Hashicorp's Vault. Both Swarm Mode and Kubernetes use a key/value store for their own settings, giving you an idea of how the community sees it as a best practice. The downside is you often want to configure the application to read and write configuration from one of these key/value stores directly. If you use something like swarm mode secrets and configs, these will be stored in docker's internal key/value database, so you get a similar solution with little effort on your part.

Not the answer you're looking for? Browse other questions tagged or ask your own question.