How to reconcile Infrastructure as Code and not storing env config in code?

Question

When developing a Web App, and trying to adhere to what is generally considered best practice, e.g. The 12 factor app methodology etc. One key concept is to keep configuration and sensitive data out of your source code - access keys, connection strings etc. Using something like .env files to set different connection strings for a dev, test and production environment.

However, another key point (not sure is is mentioned in the 12 factor app), seems to be Infrastructure as Code. I completely see the benefit of this, however, I don't see how it can be reconciled with not storing/hard coding sensative data. Surely to entirely automate the provisioning of a new environment, a series of env variables has to be specified - in code...

How can these two seemingly contradictory pieces of standard practices be reconciled?

My current approach is something like this:

1. Hosted Git Repo with however many dev branches and a master branch

2. When dev branches are merged with master branch, the CI build is triggered, this builds the project and runs unit and integration tests.

3. The git repo also has a production branch, when master is merged into production, the same build runs, but if it is successful, a CD release is created. (In my case this is a deployment to an Azure App Service)

So my three environments are the local dev machines (before a PR into the master branch in the hosted repo), the CI build server and the production app service. At the moment, locally we have a .env file which isn't checked into the git repo. On the CI server, the NODE_ENV value is set to test, so the code uses hard-coded unit test values (nothing sensitive). Then in production we manually set the connection strings etc. in the app service settings. This, I feel, is potentially an issue.

So possibly I could add an infrastructure folder to the repo, with Terraform code in it, then as part of the release process, terraform plan could see if there is anything that needs changing between the code's infrastrucutre and the real world infrastructure (and if so, execute it). But where do I store the production env variables?

Answer 1

There are, as always, a few ways to solve this.

You can use a central source to keep secrets that each server reads from ala Hashicorp Vault. While popular this is not my preferred approach as its rather complex. There are quite a few key value stores that can provide similar functionality such as AWS Parameter Store.

You can manually put data in these stores and/or store it encrypted in git. Using tools such as kops (AWS only) or ejson (provider agnostic) you can have the secrets in the same repo as the app. Then have the app either fetch all the secrets at deploy time or just the decryption key so it can load the encrypted file.

Answer 2

This is a common problem in configuration management, and while I can't speak for Chef as I haven't worked with it, I can tell you how this is solved by both Puppet and SaltStack.

Firstly, this distinction becomes a little less important with interpreted languages since there isn't much difference between including this sensitive data in a configuration file and including it in a text file that is interpreted (and I'm not aware of any compiled configuration management systems)

But secondly, both Puppet and SaltStack, so you can simply make the equivalent of a .env file and include it just as you do with your applications.

Typically this is not the recommended approach however. Instead, SaltStack recommends you use Pillars for managing this data, which are stored separately from the code (called "States"). This pillar data is uniquely calculated for each minion and sent directly from the server only to the minion only using standard SSL encryption - so one minion cannot see another's pillar data. In fact, minions can't edit pillar data and they definitely can't rewrite pillar data on the server - this data is unicast only.

Similarly, Puppet utilizes Heira for achieving something very similar. This allows you to store your code and metadata separately. You will then reference the pillar in your salt state or heira value in your manifest using a variable/function call thereby allowing your code to populate these values at runtime.