I seem to be confused about how external works. I tried:

data "external" "local_key" {
  program = [
    "ssh-keygen", "-y", "-f ~/.ssh/id_rsa"

This gives me:

failed to execute "ssh-keygen": ~/.ssh/id_rsa: No such file or directory

Which presumably happens because ~ expansion doesn't. ssh-keygen -y -f ~/.ssh/id_rsa works normally. So instead I tried invoking bash like:

data "external" "local_key" {
  program = [

  query {
    "-c" = "ssh-keygen -y -f ~/.ssh/id_rsa"

I'm still getting the same issue. I realize I can just pass the output as a var to terraform from the outside, but I'm still curious what the solution is.

  • 2
    Why insisting in using ~ where you can use $HOME? – Tensibai Feb 22 '18 at 5:36
  • @Tensibai Good point, it's just habit. :P I still don't understand why calling bash won't end up doing the expansion though. – Kit Sunde Feb 22 '18 at 6:29
  • Maybe because there's no or invalid login when Terraform execute the command? But external is supposed to be an interaction with the remote machine, not running a command on the remote machine from what I understand from the documentation. – Tensibai Feb 22 '18 at 7:14
  • I'm not sure to really understand what you're trying to achieve in fact. – Tensibai Feb 22 '18 at 7:16
  • @Tensibai External runs local programs, it doesn't imply interaction with anything remote. It's an external data source relative to terraforms execution environment, not relative to the machine terraform runs on. "external is a special provider that exists to provide an interface between Terraform and external programs." I'm just getting the public key for the local private key, the public key doesn't exist on the machine I'm running terraform on. – Kit Sunde Feb 22 '18 at 8:28

The best way to achieve what you're trying to do (generate an openssh public key from an id_rsa private key) is using the tls_public_key data provider built-in to terraform. Using the external provider should be considered an option of last resort because it introduces dependencies on the local OS which may not be portable. It's like an escape hatch.

You can use it like this:

data "tls_public_key" "example" {
  private_key_pem = "${file("~/.ssh/id_rsa")}"

Then to get the public key suitable for openssh, you can access


Lastly, if you are open to generating a new SSH key pair, you can use the terraform-tls-ssh-key-pair module which will write the keys to a new file and exposes a public_key output with the openssh public key.

| improve this answer | |

So what happens in first case is, as Dan's already said, there's no shell used and as such nothing to expand the ~. Quoting the documentation about program:

Terraform does not execute the program through a shell

On the second case, bash receive in stdin something like this:

{ "-c": "ssh-keygen -y -f ~/.ssh/id_rsa" }

And this looks like a command block for bash, but -c is not a valid command.

What could work could be this kind of program (assuming no specific input):

jq -n --arg pubkey "$(ssh-keygen -y -f ~/.ssh/id_rsa)" '{"pubkey":$pubkey}'

So something like this should work to get the key in local_key["pubkey"] if I understand the documentation properly:

data "external" "local_key" {
  program = [
    "bash", "-c jq -n --arg pubkey \"$(ssh-keygen -y -f ~/.ssh/id_rsa)\" '{\"pubkey\":$pubkey}'"

There's a need to use bash for a one liner to take advantage of command subsitution. You can also do a .sh script like:

jq -n --arg pubkey "$(ssh-keygen -y -f ~/.ssh/id_rsa)" '{\"pubkey\":$pubkey}'

And call this script in the program parameter.

| improve this answer | |

Note: this answer is based solely on documentation, I didn't try it.

From External Data Source (emphasis mine):

The following arguments are supported:

  • program - (Required) A list of strings, whose first element is the program to run and whose subsequent elements are optional command line arguments to the program. Terraform does not execute the program through a shell, so it is not necessary to escape shell metacharacters nor add quotes around arguments containing spaces.

Expanding ~ to the home directory is a capability of the shell, not of ssh-keygen and since in the 1st example ssh-keygen is invoked, it literally attempts to open a file named ~/.ssh/id_rsa. Which fails because a ~ directory doesn't exist (I'm not sure if that's what you mean by presumably happens because ~ expansion doesn't).

In the 2nd example you're passing the ssh-keygen command as a query, not as arguments to bash. Probably what's executed is not what you're expecting.

I'd try:

data "external" "local_key" {
  program = [
    "bash", "-c", "ssh-keygen", "-y", "-f ~/.ssh/id_rsa"

But I'm not sure if bash itself (or ssh-keygen for that matter) follows the External Program Protocol required (also on the above referenced page). You may need to write your own script to wrap the cmd you desire while also providing the data according to the protocol. Invoke that script as the program instead of bash or ssh-keygen.

Finally, the entire external data source page doesn't appear to suggest that export would be designed for interactive programs and ssh-keygen -y -f ~/.ssh/id_rsa asks for the user passphrase interactively. YMMV.

| improve this answer | |
  • 1
    It is not compatible, terraform give input in json and waits for a json output to be used in the query block. But I don't get the question goal... – Tensibai Feb 22 '18 at 8:26
  • Yep, that's what I suspect as well - see my 2nd to last paragraph. – Dan Cornilescu Feb 22 '18 at 8:29
  • It was a confirmation of your suspicion :) the terraform documentation is clear on this point: "The program must then produce a valid JSON object on stdout, ..." – Tensibai Feb 22 '18 at 8:34

Have you considered other options? e.g. A discussion my peers and I engaged recently was "time and place". We use mostly Ansible for deploying servers. Yet some of our team wanted to use Terraform and others argued Cloud Formation. Our discussion concluded why not let Terraform do what it does best and leave the rest to Ansible? In that within the Terraform user_data section you can call your Ansible Playbooks. My 'limited' understanding of Terraform is that its purpose is to deploy services to a limited extent and not be a full on automation drop-in replacement for services like Ansible, Chef, Puppet, etc. As a side note, Ansible has a module to handle the ssh keys generation and plenty more.

| improve this answer | |
  • I wasn't the person who down-voted you, but I think I can see why you were. Your points are valid, Terraform should be used in conjunction with a configuration management (CM) tool like Ansible. Terraform is not a replacement. However, the OP's question was about a specific function of the Terraform program. There's nothing from the OP that tells us why they are running this ssh-keygen or how its used in their pipeline. Its entirely possible that running this with terraform actually is the best place for their use case. – BoomShadow Feb 22 '18 at 16:50
  • 1
    I use terraform for orchestration and ansible configuration management, this particular issue is just about having my dropping in the initial SSH key in an automated fashion so ansible can connect. :P – Kit Sunde Feb 23 '18 at 7:11
  • @KitSunde; In my use case of Ansible we have it create the SSH key(s) for all servers and store them in out management server. Is it possible to use Ansible to create this and possibly pass that via file back to Terraform? Apologies for the inexperienced question. – Steven K7FAQ Mar 1 '18 at 17:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.