Cloud-init files are essentially bootstrap codes, that run before each startup, and can - among others - modify files, set up services, create users, etc.
Not all types of droplets support all functionalities of cloud-init, for example CoreOS uses it's own implementation, with a very limited subset of valid values.
To use this in terraform, simply provide ...
When you create an Auto Scaling group with Terraform, you can specify the user_data to be used by instances created by this ASG. Documented here - https://www.terraform.io/docs/providers/aws/r/launch_configuration.html#user_data
You can also create a single EC2 instance, and provide user_data to be used - https://www.terraform.io/docs/providers/aws/r/...
It sounds like you want to do some configuration management upstream from the terraform. I would suggest following a pattern of writing Packer templates to build and provision the AMIs before running the terraform plan and apply. By judiciously applying tags to the image name, you can write a data statement to reliably find it.
Your pipeline will be a bit ...
Hashicorp recommends using provisioners only as a last resort. They recommend using the Azure VM custom_data method, or cloud-init where available.
Ansible would likely be overkill unless you will need ongoing in-place configuration management. But there are ways to make VM2 securely accessible to Ansible or provsioners if necessary.
I feel like you are providing the answer yourself. Encode the entire cloud-init file as one-line base64 using a pre-processing script. Something like cat cloud-init.txt | base64 -w0 would work for Linux based build agents, -w0 will encode everything in one line.