How CircleCI, Terraform and AWS should interact in CI/CD Pipeline

Question

I'm new to devops and I'm have a hard time finding resources to help me with this question. My Goal is to have an automated CI/CD pipeline to launch my project.

Basically I have a GitHub repo with a Python Web application. I've also configured a pipeline in CircleCI that runs some tests in the project. That's what I've gotten to work so far.

Now, from what I've understood, this is what I would do next (This is what I'm unsure of so please correct me):

• In circleCI, generate a docker image with instructions on what kind of operating system etc and commands to pull project from GitHub repo and run database migrations etc.
• CircleCI runs terraform to launch this docker file in an EC2 Instance.

So my question is basically this:

1. Should the docker image be generated dynamically in CircleCI or static in the repo?
2. How does my source code get to the EC2 instance. Configure a git pull in dockerfile?
3. How do I tell CircleCI/Terraform how many instances I want to launch?

I know this is a question that would take too long to explain in full here. So I'm only looking to understand the basic workflow of how GitHub->CircleCI->Terraform->AWS interactive with eachother. Or any resources that explain this well.

Answer 1

There can be many correct answers, depending on the strategy you are doing. My take would be:

1. I wouldn't create a dedicated Docker image (e.g. index.docker.com/python:3.9-slim) with the application on it, but rather use the official Python Docker image, to build the app and the same to run it, so I'd package your app separately from the image that runs it. You can still compile your code as a Docker image if that's convenient and mount it as a volume. There are many benefits of this:

   • your code can be mounted as read-only, lowering the attack-surface
   • faster build (no need to add a full Python every time), faster deployment (yes, Docker layers would still work, but if your code changes, Python has updates, you might have multiple hosts eventually don't count on layering).
   • separation of concerns: just because you update your Python interpreter's patch version (e.g. for security fixes), your app don't have to change. This can be an important factor if you need to have a decision log for your releases.

2. You can use, e.g. CoreOS and use a Systemd unit file to pull down and run it for you, or you can instruct your Docker daemon via the Terraform provider to run it and set the "restart = always" so it will be started on reboots too. I'd go with the latter as it requires no knowledge on the OS and there is a single way (Terraform) to manage the whole deployment.

3. Well, if you have multiple instances, high-availability, then you either

   • need to do this in your app with a service-discovery tool (check e.g. Hashicorp Consul) or you can have a load-balancer, but that costs you money. Using load-balancers is the lazy way; using service discovery and doing the load-balancing on the client-side (app) offers you the best outcome, lesser architectural complexity, but much more work.
   • Use an orchestrator, e.g. Kubernetes, but then your whole manual deployment with Docker and Terraform is not relevant anymore because you are dealing with the deployment at a higher level. Although Kubernetes has a steep learning curve initially, it is very straightforward when it is up and running.
   • A runner-up solution could be a DNS-based solution. In this case, you manually set, e.g. 3 instances and then in your DNS zone, you set a round-robin.
   • You can't use, e.g. AWS Autoscaling Groups, because you want to manage your containers via Terraform, so if the ASG scales up an instance, somehow your Terraform needs to run. So unless you start up your containers via "user-data", ASGs doesn't make sense, but in that case, you can't manage them via Terraform because Terraform is likely aware of the container's state. Even if you manage it, it will be a fragile architecture that will bite you back when you need that high availability.

Note: You probably need at least 2 Terraform run, because the Docker Terraform provider requires access to the target Docker service via TLS/SSH or Unix socket, so your instances have to be up and ready before you run the Docker one. But that's a good idea anyway because if you add all these in your user-data/cloud-init/Ignition script, Terraform will destroy the instances, so your container updates won't be that quick.

Note 2: Do yourself a favour and if you need more than 1 instances, use an orchestrator like Kubernetes on Hashicorp Nomad.