Cloud-Init Script Won't Run?

Question

I am working through this Terraform tutorial, which boots up a Go web app on an EC2 instance, using Terraform and cloud-init. This is my Terraform config file:

terraform {
  required_version = ">= 0.13"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.0"
    }
  }
}


provider "aws" {
  region = var.region
}

data "aws_ami" "ubuntu" {
  most_recent = true

  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-*20*-amd64-server-*"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }

  owners = ["099720109477"] # Canonical
}

resource "aws_vpc" "vpc" {
  cidr_block           = var.cidr_vpc
  enable_dns_support   = true
  enable_dns_hostnames = true
}

resource "aws_internet_gateway" "igw" {
  vpc_id = aws_vpc.vpc.id
}

resource "aws_subnet" "subnet_public" {
  vpc_id     = aws_vpc.vpc.id
  cidr_block = var.cidr_subnet
}

resource "aws_route_table" "rtb_public" {
  vpc_id = aws_vpc.vpc.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.igw.id
  }
}

resource "aws_route_table_association" "rta_subnet_public" {
  subnet_id      = aws_subnet.subnet_public.id
  route_table_id = aws_route_table.rtb_public.id
}

resource "aws_security_group" "sg_22_80" {
  name   = "sg_22"
  vpc_id = aws_vpc.vpc.id

  # SSH access from the VPC
  ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }


  ingress {
    from_port   = 8080
    to_port     = 8080
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

data "template_file" "user_data" {
  template = file("../scripts/add-ssh-web-app.yaml")
}

resource "aws_instance" "web" {
  ami                         = data.aws_ami.ubuntu.id
  instance_type               = "t2.micro"
  subnet_id                   = aws_subnet.subnet_public.id
  vpc_security_group_ids      = [aws_security_group.sg_22_80.id]
  associate_public_ip_address = true
  user_data                   = data.template_file.user_data.rendered

  tags = {
    Name = "Learn-CloudInit"
  }
}

output "public_ip" {
  value = aws_instance.web.public_ip
}

My cloud-init script looks like this:

#cloud-config
# Add groups to the system
# Adds the ubuntu group with members 'root' and 'sys'
# and the empty group hashicorp.
#groups:
  - ubuntu: [root,sys]
  - hashicorp

# Add users to the system. Users are added after groups are added.
users:
  - default
  - name: terraform
    gecos: terraform
    shell: /bin/bash
    primary_group: hashicorp
    sudo: ALL=(ALL) NOPASSWD:ALL
    groups: users, admin
    lock_passwd: false
    ssh_authorized_keys:
      - ssh rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCMT6uYfhWx8jOmiGR9ryIPKcWy2ceqvyZ4
Q4+q5QTiZtlbWxbP37YZnT8uQhyjB4QRR1cjOyvGKC3Zu0Isy0eHIx2lGm/7B04b
soWWAUqhJmYWMZlivnHtJJJ4P5gnvXiRNmFg9iK07C7ClggNBAQZZHUeA5wcnvvH
T/pDkGUjMUqgLvmWRJqJM9qLT717e229F1Fyh+sYtAj08qmcFF1JCs2D33R46RQ8
YBMpQqmWLfjuJDUrjdvMu7Mv3aPpaeUWuYoC90iHR9XMeNonrtRlx21nY3CoMZ0A
OpeNl999UzyMJrsvN4qm6byK2Pc6jrEyKr9jI8SvMEGdSWgqr/Hd



# Downloads the golang package
packages:
  - golang-go

# Sets the GOPATH & downloads the demo payload
runcmd:
  - sudo su terraform
  - sudo mkdir /home/terraform/go
  - sudo chown terraform:hashicorp /home/terraform/go
  - export GOPATH=/home/terraform/go
  - go get github.com/hashicorp/learn-go-webapp-demo

After successfully applying the Terraform config, I obtain the following error when I try to SSH into this instance from the CLI using my OpenSSH private key:

Permission denied (publickey,gssapi-keyex,gssapi-with-mic).

To further investigate, I have SSH'd into the instance with AWS Direct Connect, from which I found none of the users, groups or directories had been created and the packages hadn't been installed - so it seems like the cloud-init script did not run at all. I'm guessing I can't SSH into my instance because the public key wasn't uploaded to it.

I assumed that this could be an issue with Ubuntu, so I changed the AMI to Linux 2, which resulted in the same error. Am I missing something obvious?

Answer 1

There are a few issues with this cloud-init script.

• #groups is commented out, so the list beneath it makes it invalid YAML.
• Even if we uncomment groups, - ubuntu: [root,sys] is invalid because the default user already has a group specification. If you look near the bottom of /etc/cloud/cloud.cfg on a launched instance, you should see something like:

system_info:
   # This will affect which distro class gets used
   distro: ubuntu
   # Default user name + that default users groups (if added/used)
   default_user:
     name: ubuntu
     lock_passwd: True
     gecos: Ubuntu
     groups: [adm, audio, cdrom, dialout, dip, floppy, lxd, netdev, plugdev, sudo, video]
     sudo: ["ALL=(ALL) NOPASSWD:ALL"]
     shell: /bin/bash

So you either need to remove the ubuntu line from your groups definition, or remove - default from your user definition and define the ubuntu user manually.

• Your ssh key is also pasted as invalid YAML. Keep it as a single line
• In your runcmd, sudo su doesn't work that way in a script. Instead you should sudo -u <user> for an individual user to run an individual command. Additionally, sudo isn't needed for root permissions as all user scripts run as root already.

Fixing all of these issues, we should get a resulting cloud-init script that looks something like this:

#cloud-config

# Add the empty group hashicorp.
groups:
  - hashicorp

# Add users to the system. Users are added after groups are added.
users:
  - default
  - name: terraform
    gecos: terraform
    shell: /bin/bash
    primary_group: hashicorp
    sudo: ALL=(ALL) NOPASSWD:ALL
    groups: users, admin
    lock_passwd: false
    ssh_authorized_keys:
      - ssh rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCMT6uYfhWx8jOmiGR9ryIPKcWy2ceqvyZ4Q4+q5QTiZtlbWxbP37YZnT8uQhyjB4QRR1cjOyvGKC3Zu0Isy0eHIx2lGm/7B04bsoWWAUqhJmYWMZlivnHtJJJ4P5gnvXiRNmFg9iK07C7ClggNBAQZZHUeA5wcnvvHT/pDkGUjMUqgLvmWRJqJM9qLT717e229F1Fyh+sYtAj08qmcFF1JCs2D33R46RQ8YBMpQqmWLfjuJDUrjdvMu7Mv3aPpaeUWuYoC90iHR9XMeNonrtRlx21nY3CoMZ0AOpeNl999UzyMJrsvN4qm6byK2Pc6jrEyKr9jI8SvMEGdSWgqr/Hd

# Downloads the golang package
packages:
  - golang-go

# Sets the GOPATH & downloads the demo payload
runcmd:
  - mkdir /home/terraform/go
  - chown terraform:hashicorp /home/terraform/go
  - sudo GOPATH=/home/terraform/go -u terraform go install github.com/hashicorp/learn-go-webapp-demo@latest

Answer 2

Maybe it does not matter but also try to add "${}" around terraform's file function before rendering:

data "template_file" "init_script" {
  template = "${file("../scripts/add-ssh-web-app.yaml")}"
}