The context of the question is Secrets management.

As the title says, how do you change a secret that half of your production applications use right now?

  • What did you try?
    – 030
    Commented Mar 18, 2018 at 15:59
  • @030, I've tried out Hashicorp's Vault and recently tried AWS Parameter store. Commented Mar 19, 2018 at 8:13
  • Could you clarify the question of what you mean by 'half of your production applications use right now' ? Are you looking for a way to update only half of your production apps and leave the others as they are, or are you saying you have configs in production that don't match each other and you would like to force a consistence usage of secrets across all of the apps?
    – BoomShadow
    Commented Mar 19, 2018 at 15:16

2 Answers 2


This question ties in heavily with your other question with where the secrets are set and where they retrieved from.

I assume your question is asking about 'canary deployments' where you change the config for only a small portion of your apps, to test things out before deploying everywhere.

If you follow the best practice of using the 12 Factor app's recommendation of 'environment variables', then changing secrets becomes very easy. At the very worst, you could log into the production server, update the ENV VAR, and then restart your app manually. In Kubernetes, this is even easier, as you can edit the ENV variables from the UI, and then restart the pod.

However, you don't want to be doing that manual process if you can avoid it. If your Configuration Management tool sets the ENV VAR, then you can tell it to update a specific group of servers only. If your build server injects the ENV vars, and it supports canary deployments, then you can easily just do a canary deployment like normal.

  • I'm guessing that by "secret that half of your production applications use right now", OP is referring to something like a password that over half (arbitrary number) of his/her production env is using. Either way, your recommendation of environment variables still applies, and in my opinion is a great approach :)
    – Preston Martin
    Commented Mar 19, 2018 at 18:18

i wouldn't 'change' the secret at least in the short term but roll out a second secret with the same permission set as the original and roll that out through your configuration management or secrets management. ideally following @boomshadow 's answer. once the new credential is fully rolled out invalidate the old one/s repeat as necessary.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.