This is a very broad question about methods and advice regarding environment variables / structure. But ultimately I'm looking for answers for the very specific question of 'How should I store my environment variables?'
Firstly some clarifications:
- An environment to me can be from 3 to 10 servers and is a way of containing a specific customer's infrastructure.
- Inside each environment there are a few variables which are mostly automatically generated from a few key inputs (name, size etc).
As it stands right now we're storing all of our environment variables in a structure like so:
<playbook>.yml # Various playbooks for deployment
roles/windows # Ansible role for Ubuntu
roles/ubuntu # Ansible role for Ubuntu
config/hosts/<name>.yml # Ansible inventory
config/hosts/vars/<name>.json # Environment specific variables
Right now the config is initialised as a submodule in the git repository above. As the variables file changes rather frequently this has caused issues with the data changing, once, twice or even three times between commits making changes increasingly difficult to trace.
As I personally see it going forward we should be looking to store all our customer variables in a centralised / scalable way and then hook into it with a dynamic inventory with ansible.
I understand there are a few technologies which seem to do a part of what could be required such as Consul but they seem to work best in an environment which serves one big application rather than lots of smaller slightly differing ones.
I essentially see us having to write an inventory script and then just shove all of our data into some not-for-purpose built database and then keep going on as if nothing changed. I see this conceivably as a way to potentially trim down on a lot of the data we currently store and perhaps look into different ways of storing the data rather than just scaling up what serves it again.
I'm hoping somebody has some kind of experience in implementing infrastructure as code when having to deal with lots of smaller environments as opposed to one, two or three huge ones.