How can I deploy a Kubernetes cluster onto OpenStack with Terraform?

Question

I know OpenStack has a supported method of installing a Container Container Orchestration Engine (COE) like Kubernetes using Magnum. I can not use Magnum because I am not an administrator on OpenStack and we do not have the "Service Endpoint" for Magnum installed.

---

Terraform has the ability to provision resources on OpenStack: I have this working. Is there any easy way to provision multiple resources and to configure a Kubernetes cluster on top of OpenStack from a non-admin level of permissions? I'm open to using other tools like Ansible to get this job done. I'm just wondering if this is possible as a regular non-admin of OpenStack to easily set up a Kubernetes cluster from the resources I can provision?

Answer 1

Most likely the best path forward is to provision the VMs using Terraform, and the rest with Ansible.

OpenStack can be set up a number of ways, but I believe the only hook in the terraform provider that would allow you to execute code on the node would be via cloud-init user data (bash script).

What I would do is provision nodes with terraform and have it provide outputs, have python generate Ansible hosts and vars from those outputs, and then run Ansible.

What flavor of k8s are you looking at? k0s, k3s, microk8s, etc?

Answer 2

Terraform + Ansible could be the best path forward.

Terraform also has local and remote exec provisioners when provisioning VMs. Local exec runs scripts from the local machine you're deploying from and remote exec spins up an on demand SSH server using a local public key for authentication to run commands on the deployed VM for configuration (eg setup.sh)

https://developer.hashicorp.com/terraform/language/resources/provisioners/remote-exec

resource "aws_instance" "web" {
  # ...

  # Establishes connection to be used by all
  # generic remote provisioners (i.e. file/remote-exec)
  connection {
    type     = "ssh"
    user     = "root"
    password = var.root_password
    host     = self.public_ip
  }

  provisioner "remote-exec" {
    inline = [
      "puppet apply",
      "consul join ${aws_instance.web.private_ip}",
    ]
  }
}