0

My team uses Terraform to deploy the necessary infrastructure. Currently, our stack is composed of: AutoScalingGroup fronted by a Load-Balancer, and a Route53 DNS record that points to this LB;

We would like to extend the stack to include a Cloudfront distribution that points to the LB, and make the Route53 DNS record point to the CDN.

I would like to be able to:

  • spin up Cloudfront and change the DNS record to point to the CDN, without causing any downtime;
  • tear down the CDN, change the DNS to point to the load-balancer, without causing any downtime;

The current configuration is as follows:

resource "aws_lb" "alb" {
  name                       = "${var.alb-name}"
  ...
}
resource "aws_cloudfront_distribution" "cdn" {
  count = "${var.use_cloudfront != "false" ? 1 : 0}"
  ...
}
resource "aws_route53_record" "www" {
  count = "${var.domain-name != "" ? 1 : 0}"

  depends_on = ["aws_lb.alb", "aws_cloudfront_distribution.cdn"]  

  zone_id = "${data.aws_route53_zone.primary.zone_id}"
  name    = "${var.domain-name}"
  type    = "A"

  alias = {
    // we use concat because 'count' makes the response of the resource a list. 
    // link to similar issue: https://stackoverflow.com/questions/45654774/terraform-conditional-resource
    name                   = "${var.use_cloudfront == "true" ? element(concat(aws_cloudfront_distribution.cdn.*.domain_name, list("")), 0) : aws_lb.alb.dns_name}"    
    zone_id                = "${var.use_cloudfront == "true" ? element(concat(aws_cloudfront_distribution.cdn.*.hosted_zone_id, list("")), 0) : aws_lb.alb.zone_id}"
    evaluate_target_health = true
  }
}

The current behavior is as follows:

  • the spin-up of the CDN, followed by re-routing the DNS record to point to the CDN, does not lead to any downtime;
  • the tear-down of the CDN, followed by re-routing the DNS record to the load-balancer, does cause downtime;

I am aware that the depends_on = ["aws_lb.alb", "aws_cloudfront_distribution.cdn"] on the Route53 record might not be the best choice to create the desired behavior. However, I am not sure how to reach the desired behavior using terraform.

I've also added the Terraform output to a Gist file: https://gist.github.com/rafaelmarques7/03ec6e576faff81ef36d7fc878e85230

I am wondering if anyone has any suggestions to reach the desired behavior. Let me know if the provided information is insufficient. Thank you for your help, it is appreciated!

0

Modelling these permutations as a single module with lots of parameters is possible as you've seen, but it's often not an ideal way to model things because this sort of careful rollout strategy tends to require control over individual steps, rather than rolling everything all at once.

The Terraform team recommends using module composition instead. That means to split the problem into multiple smaller modules that the calling module can then combine to achieve the desired result.

In your specific case, it looks like there are three different building blocks: a load balancer, an optional Cloudfront distribution, and a DNS record. I'm going to show some examples of module composition using Terraform 0.12 features. If you're still using Terraform 0.11 then you can still follow a similar pattern, but Terraform 0.11's expression handling is more limited so the details would look different.

When breaking down this problem, my first step would be to identify what information needs to flow between these components. In this case:

  • The load balancer has some arguments of its own, but it doesn't depend on the other components at all.
  • The optional Cloudfront distribution needs to know the hostname of the load balancer.
  • The Route53 record needs a record name and zone id to populate an alias block; they could be for either the load balancer or the Cloudfront distribution depending on which of the two architectures the calling module is following.

In the full architecture where Cloudfront is used, these modules might be composed like this:

module "load_balancer" {
  source = "./modules/load-balancer"

  # ... load-balancer specific arguments
}

module "cloudfront" {
  source = "./modules/cloudfront"

  target = module.load_balancer.dns_name
}

module "dns" {
  source = "./modules/route53-record"

  alias = module.cloudfront.route53_alias
}

In the load-balancer-only architecture, we remove the cloudfront module and connect the DNS directly to the load balancer:

module "load_balancer" {
  source = "./modules/load-balancer"

  # ... load-balancer specific arguments
}

module "dns" {
  source = "./modules/route53-record"

  alias = module.load_balancer.route53_alias
}

Terraform 0.12 allows us to define that alias variable on the dns module with a specific object type, to easily pass the alias settings between modules as a single object value:

# (in the route53-record module)
variable "alias" {
  type = object({
    name    = string
    zone_id = string
  })
}

resource "aws_route53_record" "example" {
  # (other arguments as in your example)

  alias {
    name                   = var.alias.name
    zone_id                = var.alias.zone_id
    evaluate_target_health = true
  }
}

The idea here is that both of the other modules follow this conventional structure so that the route53-record module can be compatible with either of them without caring which one is used. This would then correspond with outputs in each of the other modules following the same structure:

# in the load-balancer module
output "route53_alias" {
  value = {
    name    = aws_alb.example.dns_name
    zone_id = aws_alb.example.zone_id
  }
}
# in the cloudfront module
output "route53_alias" {
  value = {
    name    = aws_cloudfront_distribution.example.domain_name
    zone_id = aws_cloudfront_distribution.example.hosted_zone_id
  }
}

With this different design, we can address your original use-case by giving control to the author of the calling module to drive the switch between these two models. In order to avoid downtime when switching from Cloudfront back to the load balancer, we need to repoint the DNS first and then destroy the Cloudfront distribution only when it is no longer used. So we can do that in two steps, first by modifying the first example above to point the DNS record at the load balancer while retaining the Cloudfront distribution:

module "load_balancer" {
  source = "./modules/load-balancer"

  # ... load-balancer specific arguments
}

module "cloudfront" {
  source = "./modules/cloudfront"

  target = module.load_balancer.dns_name
}

module "dns" {
  source = "./modules/route53-record"

  alias = module.load_balanccer.route53_alias
}

Then, once the Cloudfront distribution has become idle, remove that module altogether at your leisure and apply again to clean it up.

Composition is the primary way we model relationships in Terraform, so it's most natural to use composition when working with modules too, rather than building big modules with lots of conditionals to handle every permutation. Composition allows flexibility in how to make changes to the infrastructure in future so that the maintainer of the top-level module can be the one to decide what tradeoffs to make, such as making smaller changes to avoid downtime.


I used modules above because the original question was about modules, but I'd note also that after this decomposition we're left with one module per resource whose name is the resource type, which is one of the "smell tests" in When to write a module?.

If decomposing these into separate modules ends up just creating modules that are thin wrappers around single resource blocks then it's usually better to just eliminate the module altogether and just use the resource types directly.

However, if one or more of the modules still raises the level of abstraction (e.g. by providing hard-coded good values for most of the resource arguments and only exposing a small subset of them for customization) then that module in particular may still be warranted.

| improve this answer | |
  • Hello Martin. Thank you for this very mindful, interesting and eye-opening response. I have read it twice already, but I still need a couple more times to fully get this through my head hehe. I appreciate that you explained everything in such detail, and the design your proposing makes a lot of sense. Now it's time for me to take some lessons from my mistakes. Thank you for this answer! – Rafael Marques Jun 14 '19 at 9:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.