Is there an idiomatic way to create reusable Packer templates?

Question

I'm creating about 10-12 Packer templates which almost all work the same way. Same builder (Amazon EBS), with some small variations in AMI names, and almost the same provisioner (Ansible Remote) - sometimes with additional variables and sometimes multiple playbooks.

But, much of the Packer templates remain the same.

I know I can use variables to get some of the changeable values into one place in the file. But I'm keen to cut down on all of the copy-pasta in the templates.

I could do this with a wrapper shell script utilising jq or something. But each complexity I add gives something else for others to learn. Try as I might, I can't seem to find a 'blessed' idiomatic way of doing this with Packer.

Is there an idiomatic way of doing this - or at least a generally accepted way?

Answer 1

So far, I've taken to defining the default builders and provisioners we use, with user variables interpolated in where required, eg:

 "type": "amazon-ebs",
 "instance_type": "t2.micro",
 "ami_name": "{{ user `ami-name` }} 

I've got files called eg. packer-builders.json and packer-provisioners.json, and then all the separate Packer templates in their own directories.

I've then written a quick Bash script that takes the name of the template one wants to build, finds all the packer-*.json files, and uses jq --slurp to merge them in (eg. jq --slurp '.[0] * .[1] * .[2]' $LIST_OF_FILES) with the supplied template at the end. It saves the output and sends that along to packer build.

As a bonus, I'm also using jsmin before hitting jq so that I can include comments in any of the JSON files.

It works well, but I'm not a massive fan of it because it's custom and it adds an additional barrier for entry for others coming across it (much more to consider than just packer build something.json!

So I'm really keen for additional answers here if there's a better way to do this.

Answer 2

Packer has taken an active decision not to build in such capabilities. This problem is much better solved by other tools such as jq, etc. We think the best way to leverage Packer is to wrap it if you need more ways to create the template.

It's very easy to create json in any programming language which gives you unlimited expressiveness when creating templates. One great way to manipulate templates is to define them in jinja2 and preprocess it with Python.

Answer 3

Utilizing the HCL2 syntax is definitely the correct approach here. You can use the different building blocks to define different source blocks and then assign sources to the builders and even conditionally execute the provisioners and post processors on specific sources using only/except.

Using this approach you can easily accomplish what you are looking for. Here is a rough example adapted from the hashicorp documentation with multiple sources and ansible:

source "amazon-ebs" "first-example" {
    ...
}
source "amazon-ebs" "second-example" {
    ...
}
build {
  name = "my_build"
  sources [
    "source.amazon-ebs.first-example","source.amazon-ebs.second-example"
  ]

# I run on both sources 
provisioner "ansible" {
        playbook_file = "common.yml"
  }

# Only runs on first example
provisioner "ansible" {
    only = ["source.amazon-ebs.first-example"]
    playbook_file = "playbook-a.yml"
  }

# Never runs on second example
provisioner "ansible" {
    except = ["source.amazon-ebs.second-example"]
    playbook_file = "playbook-b.yml"
  }
}

Depending on your source blocks and the builder being used, this opens up the ability to template multi-cloud, arch, distribution and purposed images from the same packer build command.

Answer 4

This is what source blocks are for in HCL2. I've been using this approach recently (define builder in a source block, then use it from various build blocks, filling in the variables as needed) and believe this to be the new best practice / idiom.