I want to add eight public keys via instance metadata to avoid adding them manually (i.e.: ssh to VMs, pasting the keys to .ssh/authorized_keys, etc.).
I added the keys in Terraform (four distinct keys for two users) using the metadata attribute of the google_compute_instance:

resource "google_compute_instance" "host" {
  count         = var.number_of_hosts
  // vm details...

  metadata = {
    "ssh-keys" = <<EOF

I ran terraform apply. I opened the GCP console and clicked on one of the deployed machines. In the "Details" tab, I can see all eight keys in the SSH Keys tab. Now, when I ssh from my local computer, i.e., ssh user2@EXTERNAL_IP (I deliberately started with user2, not user1 - not a typo) and then cat ~/.ssh/authorized_keys, I can only see the following:

user1 : key1
user2 : key4

Thus, I can't ssh to VM2 because the public part of the key pair that USER 2 has access to is not ~/.ssh/authorized_keys even though it is declared in the instance metadata.

On the other hand, when I do user1@EXTERNAL_IP and cat ~/.ssh/authorized_keys, I can see:

user1 : key1
user2 : key4
user1 : key1 (duplicate)

Since the private key that corresponds to user1 : key1 is there, I can ssh to VM2 successfully.

What baffles me:

  1. Why are not all keys declared in the instance metadata added to the authorized_keys?
  2. Why is there a difference in the content of the authorized_keys depending on the user?
  3. Where does the duplicate come from?

Edit - some additional information:

  1. the image used - ubuntu-minimal-2004-focal-v20230427
  2. ssh_config (only uncommented lines):
Include /etc/ssh/ssh_config.d/*.conf
Host *
SendEnv LANG LC_*
HashKnownHosts yes
GSSAPIAuthentication yes
  1. sshd_config (only uncommented lines):
Include /etc/ssh/sshd_config.d/*.conf
PasswordAuthentication no
ChallengeResponseAuthentication no
UsePAM yes
X11Forwarding yes
PrintMotd no
AcceptEnv LANG LC_*
Subsystem       sftp    /usr/lib/openssh/sftp-server

  • What software are you using in your VM image to interpret the metadata and generate those authorized_keys files? I expect it's that software which is making the decision about which files to write and what to write into each of them. From Terraform's perspective and GCP's perspective this is just an opaque string, so how it gets interpreted is up to the software running inside your VM. May 25 at 15:51
  • I edited my answer with the sshd_config and ssh_config, but I'm not sure that is what you meant with "software are you using in your VM image to interpret the metadata and generate those authorized_keys files". May 25 at 17:21
  • The way this metadata concept works is that Google Compute Engine just exposes an internal API that returns the metadata values and then some software running in your VM -- typically during the boot process -- will make requests to that API to fetch the data and then reconfigure your system based on what it reads. It's that software that would be responsible for generating these authorized_keys files. If you're using a generic Linux distribution image then perhaps you can name that distribution so we can find docs about it. May 25 at 23:25
  • Here's the info from the description tab in the GCP console: Canonical, Ubuntu, 20.04 LTS Minimal, amd64 focal minimal image built on 2023-04-27, supports Shielded VM features May 26 at 6:19
  • According to Ubuntu's documentation, official Ubuntu cloud images since 18.04 use CloudInit for boot-time initialization. That suggests that this is a CloudInit question, and so I'm going to add a label for that in the hope that it will be more visible to those who have CloudInit expertise. May 26 at 15:19


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.