3

There is a new generation of tooling developed that tries to integrate different IaaS providers (Azure/AWS/GCE/DigitalOcean) into one "infrastructure codebase". It's dubbed "Cloud Native" and it's supposed to result in "one tool to rule them all". There's Terraform from HashiCorp, InfraKit from Docker, Juju from Canonical, independent Foreman, Pulumi and others.

It's easy to get lost. Which of these tools is best suited to IaaS on premises? Let's suppose you have a bunch of metal servers and you want to provision some VMs there. Let's suppose you don't want VirtualBox/VMWare/ESX/Xen, you want KVM. And on top of that an obvious choice is libvirt. To manage multiple hypervisors we need to connect to the libvirt daemon remotely using connection URI. For KVM we can use qemu+ssh. Let's suppose we set those up using PXE boot. Now we have a bunch of qemu+ssh:// links along with ssh access.

Now we need a central component on top of that to manage resources and some tool that will provision virtual hosts according to some declarative specification.

IaaS on premises using libvirt qemu kvm

On top of those VMs I want to run containers, so automatic setup of container orchestration cluster is desired. In addition, some way of attaching storage is desired. At least directly attached block devices or fs mounts.

Any suggestions?

Note: Found a similar question on SO, but for Xen.

Clarification: "best suited" needs metrics. So these are the metrics I'm looking to optimize:

  • Zero closed source LOC
  • Number of tools if not "All-in-one"
  • UI actions to setup 3 master + 5 workers k8s/swarm cluster
    • number of commands (for CLI tool)
    • number of clicks (for Web based/GUI tool)
    • length/size of the config file
  • UI actions to install the tool
  • UI actions to maintain configuration (preferably none, self-healing infrastructure)
Improve this question
0

2 Answers 2

Reset to default
3

"best suited" is the question. How do you measure this? what are the metrics and why do you choose them? I think you are looking for some kind of "tools combination", earlier tested on production by someone else's if preferred. What you want to do is try some opensource/free private cloud on-premises and run IaC on this private environment. You can test tools forever. How do you compare those tools? I believe all of them works fine as long as there is not any inconsistency problem. I think there is a missing/main part in your search when it comes to IaC. Because of few options for that missing angle as well. I think it is better to create a chart for all of us as you are going this process in order to benefit people here. Do you have a working environment? If so could you please detail all the tools you used to create this environment?

Some vendors run open source projects. How about them?

The Cloud Native Computing Foundation seeks to drive adoption of this paradigm by fostering and sustaining an ecosystem of open source, vendor-neutral projects.CNCF Cloud Native Definition v1.0

I started to wonder if this is cloud-native then what is it that is not cloud-native?

I think this is what you are looking for. The distinction between provider and provisioner is blurred. A workflow automation tool should be doing what cloud orchestrator with GUI does. Terraform makes a distinction between provider and provisioner but Cloudify removes them as I see it. For an architect designing the whole workflow with GUI might be useful to reach the desired state. Writing desired state for which you do DRY tests already beforehand or showing with graphs. There is something you know already where it goes since you already tested, seeing it flowing in graphics workflow makes sense? Then go for it. To me it is painting the history I do not see the benefit.

There are many ways to break down the types of automation, whether it’s imperative versus declarative, or orchestration versus desired configuration state.

Most orchestrators are fully CLI based, or offer very limited GUI operations. A distinct advantage here is that Cloudify enables you to one-click your way through architectural relationship diagrams. See: Terraform vs. Cloudify.

Providers

A provider is responsible for understanding API interactions and exposing resources. Providers generally are an IaaS (e.g. AWS, GCP, Microsoft Azure, OpenStack), PaaS (e.g. Heroku), or SaaS services (e.g. Terraform Enterprise, DNSimple, CloudFlare). See: Providers.

Provisioners

Provisioners are used to execute scripts on a local or remote machine as part of resource creation or destruction. Provisioners can be used to bootstrap a resource, cleanup before destroy, run configuration management, etc. See: Provisioners.

Improve this answer
3
  • I'm looking for "All-in-one" tool. So if this is possible with combination of tools the metric to measure is "number of tools used to setup k8s cluster from scratch". If this is not possible with one CLI command, the next metric is "number of commands needed". If there's no CLI, the metric is "number of clicks". The next metric is "number of clicks/commands to install the tool".
    – Vanuan
    Commented Oct 20, 2018 at 2:06
  • Added metrics into question.
    – Vanuan
    Commented Oct 20, 2018 at 2:16
  • What you are looking for is in your "clarification" words and also in the title of the link you gave. I think you focus too much on tools and lose the sight of your problem. I do not want to go further because I do not know what kind of problem you are facing. I have to create your environment and try. You want to deploy clustered-containers on some cloud. Inconsistency would be a problem in most cases.
    – hakkican
    Commented Oct 22, 2018 at 11:34
1

Here's what I've tried:

Terraform + libvirt

There's unofficial libvirt provider for terraform. It works, but there are a few gotchas:

Docker machine

Docker machine has unofficial kvm driver.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.