How do I use the remote-exec provisioner with Terraform?

Question

I am using Terraform 0.14 and AWS. I am trying to write a .tf file that will invoke a remote-exec command via Terraform's provisioner.

This code does not work:

resource "aws_instance" "example" {
  ami           = "ami-1234"
  instance_type = "t2.large"
  key_name      = "foobar"
  user_data     = local.cloud_config_config

  provisioner "remote-exec" {
    inline = ["sudo yum -y install coolprogram"]
  }
}

The above will create the EC-2 instance. But it won't run the inline command. After I run "terraform apply" with the above content in my .tf file, I see "Error: host for provisioner cannot be empty"

I tried this .tf file variation, but I get an error about the variable:

resource "aws_instance" "example" {
  ami           = "ami-1234"
  instance_type = "t2.large"
  key_name      = "foobar"
  user_data     = local.cloud_config_config

  provisioner "remote-exec" {
    inline = ["sudo yum -y install coolprogram"]
  }
  connection {
    host     = "${var.host_name}"
  }
}

The hostname will be created dynamically. Terraform's documentation does not mention a hostname. I have tried different variables.tf files.

But I always get something like this:

aws_instance.example (remote-exec): Connecting to remote host via SSH... aws_instance.example (remote-exec): Host: example aws_instance.example (remote-exec): User: root aws_instance.example (remote-exec): Password: false aws_instance.example (remote-exec):
Private key: false aws_instance.example (remote-exec): Certificate: false aws_instance.example (remote-exec): SSH Agent: false aws_instance.example (remote-exec): Checking Host Key: false aws_instance.example (remote-exec): Target Platform: unix

How do I use the remote-exec provisioner on an EC-2 server?

Answer 1

here is a complete example:

resource "aws_key_pair" "my_key" {
 key_name   = "my_key"
 public_key = file(pathexpand("~/.ssh/id_rsa.pub"))
}

resource "aws_instance" "example" {
  ami = my_ami
  instance_type = "t2.micro"
  key_name = aws_key_pair.my_key.key_name
  ebs_block_device {
    device_name = "/dev/sda1"
    volume_size = 50
  }
  provisioner "remote-exec" {
   inline = ["my_command"]
  }
  connection {
   host        = coalesce(self.public_ip, self.private_ip)
   agent       = true
   type        = "ssh"
   user        = "My_user_name"
   private_key = file(pathexpand("~/.ssh/id_rsa"))
  }
}

so I assumed that you have your public and private key located in your home directory in .ssh otherwise you need to hard code the path. and you need to make sure that your IP is allowd in the AWS security group.

Answer 2

Terraform Provisioners are a last resort, so I'd encourage you to think about other options first.

In this case, your example suggests that you're already using cloud-init and so if at all possible I'd suggest modifying that cloud-init configuration to include a request to install the package you want to install. If you are using the cloud-config YAML format then you can use the Package Update Upgrade Install module to get a similar effect to your yum install command line:

packages:
 - coolprogram

---

However, if you've eliminated all other options except provisioners then the typical way to use remote-exec with an EC2 instance is to pass one of the instance's own IP addresses as the hostname, which you can do by using the special self object in the connection block to refer to the attributes of the object that the provisioner is running against, like this:

resource "aws_instance" "example" {
  ami           = "ami-1234"
  instance_type = "t2.large"
  key_name      = "foobar"
  user_data     = local.cloud_config_config

  connection {
    host = self.private_ip
  }

  provisioner "remote-exec" {
    inline = ["sudo yum -y install coolprogram"]
  }
}

The above example uses self.private_ip, which will work only if the host where you are running Terraform is running in or able to connect to the VPC where you're deploying this instance. Another option is to use self.public_ip to use the public IP address, but that will work only if your VPC has an Internet Gateway and your instance belongs to a security group that can accept incoming SSH connections from the internet.

In addition to supplying a suitable IP address, you'll also need to generate and assign an allowed SSH public key to the EC2 instance and provide the corresponding private key in the connection block, using the private_key argument. (Or one of the other authentication strategies that a connection block supports.)

These connection and authentication complexities are, as you hopefully saw in the Terraform documentation I linked earlier, one of the main reasons why provisioners are a last resort. Using cloud-init instead will avoid this because cloud-init runs automatically on system boot and retrieves the user_data string using a private API accessible from the instance's private network interface, and so it doesn't need any outside connectivity, to start any listening services, or to generate any new credentials.

Answer 3

As it was pointed out by Martin, remote-exec provisioner may not be able to connect to the target EC2 instance because of networking or security issues.

If the EC2 instance is SSM managed, you can use AWS Systems Manager Run Command to execute command on the instance.

Resource ssm_command from 'pbobov/ssm' provider can be used to run SSM commands by Terraform. It was specifically designed as a workaround for using remote-exec provisioner.