Is terraform good option for data management

Question

I have complex stack which consists of infrastructure, application and config parts, which are deployed to datacenters around the world. We have custom inhouse configuration system, based on database which is replicated elsewhere.

We have loads of configuration, which depends on our infrastructure - i.e. database connection strings, application specific configuration etc. Right now all this config is managed by hand, and drifts a lot. It's hard to say which config is still required and which can be decommissioned, it's hard to ensure exactly the same changes go from test environment to production, and that all environments were updated.

I wonder how can I improve this situation. I would like to have single source of truth in form of git repository, which can then be deployed to multiple datacenters (with some templating of variable substitution).

When I read this requirements, I think terraform with some kind of sql CRUD provider would be perfect tool:

• stored in git
• infra is already defined there, so pushing stuff like database connection strings is easy
• can review changes
• no manually configuring data in the system
• easy replication across environments

System based on database has to stay (too much code depends on that). Of course there is other data in the database, which changes a lot or is customer specific, and would not be managed by this setup.

Does it make sense to use terraform in such way?

Other tools I've checked:

• Ansible
• Salt
• Chef
• Puppet

All these tools were built with similar setup, however I see following downsides:

1. Would have to push infrastructure identifiers into these tools (which is repetitive and error-prone)
2. These are complex tools, which would have to be deployed alongside terraform
3. These tools do not track state and changes, so it's not possible to see what I'm changing (i.e. terraform plan)
4. When infrastructure is provisioned there is separate bootstrapping process to register in configuration tool

Answer 1

Terraform from high level is called IaC tool (Infrastructure as code) tool; So it comes in most handy when you use it in its primary function, providing infrastructure.

The part of configuration should be handled by configuration management tools; as you mentioned like ansible, chef, puppet ect..

By my experience, in one product, we are using terraform and ansible, hand to hand; it works perfectly fine, and not just it does the job, it is easy to read, understand and maintain/change/organize. Mixing IaC and Configuration management tools in general is so good to maintain your whole platform.

However, In some other product I worked on before, when they originally designed the product, they tried to use terraform as you say in every aspect, building and configuring, since you theoretically can execute whatever script with terraform plugins like null_resource; so it is doable, but on the contrary of native configuration management tools: it is hard to understand, maintain and change.

Of course all of the above depends on the complexity of each system and its design and components, but my advise, if you want configuration management, go for configuration management tools.