46

I would like to use the Terraform MySQL Provider to keep a list of mysql users and grants handy for creating new test environments. The .tf and .tfstate files both seem to want to store the MySQL passwords in plaintext.

Concerning .tf:

It is my understanding that .tf files live in revision control and are maintained by a team. How does that practice differ when secrets are in the .tf? It is possible to encrypt these values at all?

Concerning .tfstate:

I can store the .tfstate securely somewhere after running Terraform apply, but it would be preferable for this use case to not store it at all?

22

Terraform supports adding an additional file with variables during invocation.

documentation: https://www.terraform.io/intro/getting-started/variables.html#from-a-file

We are using that feature to provide a secrets.tfvars file on each invocation of Terraform. We also use a script to wrap the command so that its invocation is consistent, and all team members avoid having to make the same mistakes. The wrapper synchronizes .tfstate with S3 before an execution, and pushes .tfstate back to S3 at the end. I also hear of people doing the same thing with state stored in Consul, even adding a kind of semaphore in consul to prevent two people from starting Terraform at the same time.

When you avoid setting a default value in a variables.tf file, it forces the user to input the value. It can be either entered manually or using the -var-file command option like described above. Not setting a default on your secrets is a good way to enforce changes that require a change in secrets.

The secrets.tfvars file is a symbolic link to one of the files with secrets which are not stored in version control. We have several, one per environment, like so secrets-prod.tfvars, secrets-dev.tfvars, secrets-stg.tfvars, etc...

An even better practice would be to generate these secrets files during the wrapper script based on data in Vault or some other way to share secrets. Since currently when the format of secrets changes, or secrets themselves, we need to communicate it to the team outside the version control channel - and this doesn't always work well, to be honest. But secrets do change infrequently.

8

We avoid terraform handle our secrets. Even if you manage to inject secrets by a var file "secrtes.tfvars" as pointed out above, these secrets will be stored in your terraform (remote-)state.

You can protect remote-state files by using e.g. S3 authorization, or you can gitignore local state files but we decided not to rely on this kind of protection.

6

If you're on AWS, then have a look at "The Right Way to Manage Secrets" by Segment.io on the AWS Blog. We advocate using chamber to all of our customers for managing secrets. It works by leveraging the AWS Systems Manager Parameter Store (SSM) together with KMS keys. This ensures secrets are encrypted at rest (and in transit), secured with IAM, auditable with CloudTrails, and only exposed as environment variables at run-time.

After configuring chamber and setting up the KMS key, we write the secrets to the parameter store.

chamber write db TF_VAR_DB_USER foobar
chamber write db TF_VAR_DB_PASS secret

Then use those secrets when you call terraform.

chamber exec db -- terraform plan

This assumes you've defined a variable called DB_USER and DB_PASS in your HCL code.

For example, you could add this to variables.tf

variable "DB_USER" { }
variable "DB_PASS" { }

NOTE: chamber will always export environment variables in uppercase

We provide a terraform module called terraform-aws-kms-key to make provisioning the KMS key easy. Check out our detailed documentation with examples of how to use chamber with multiple namespaces as well as how to use chamber with terraform to manage secrets. See our complete reference example for provisioning chamber dependencies.

As for .tfstate, you bring up a really good point about the existence of plain-text secrets in the state file. There's really no way around this. In order for terraform to calculate changes to build a plan, it needs to know the "before" and "after" state. For this reason, we recommend using an encrypted S3 bucket with mandatory versioning. Use the terraform-aws-tfstate-backend module to provision a bucket and DynamoDB locking table according to best practices.

  • This is highly tied to AWS services, which the question doesn't mention and sounds not really an answer for on premise infrastructures or any other cloud. – Tensibai Jul 27 '18 at 8:21
  • @tensibai, you are correct. The original question does not provide enough information to ascertain the operating platform in order to make the best recommendation. Every platform will be different depending on platform capabilities. On prem or baremetal users may want to consider using a combination of Vault and Terraform Enterprise. The scope of implementing that will be much, much greater. :) – Erik Osterman Jul 27 '18 at 20:42
  • I already use AWS Secrets Manager and dont want to use chamber and Parameter Store. Is it possible to do this same thing with Secrets Manager too? – red888 Jan 21 at 16:19
3

To import secrets into .tf files, you can also use an external data source. This could be e.g. a script which decrypts your secrets.

2

I looked at a few different ways but I particularly liked git-crypt for an adhoc thing to do before implementing something bigger like Vault.

  • 2
    to thoe who downvoted, please explain why. – jottr Oct 2 '17 at 19:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.