Scheduling collisions possible in IaC - Infrastructure as Code - solutions vs. software defined data center?

Question

To clarify the terms first,

• Infrastructure as Code (IaC) can be delivered with upcoming technologies like Docker Swarm, Kubernetes, OpenShift, OpenStack and possiby some others run on bare metal. - You can code the infrastructure.

• Software defined data center (SDDC) are typically enterprise and mostly more traditional solutions from IBM, VMWare, recently also Nutanix any many others. -You can define the infrastructure through some sort of UI.

Correct me if I am wrong.

Then, typically, my assumption/observation larger companies (much larger than a s startup, much smaller than Google) would provision you with some SDDC resources and there you can play with IaC.

In such a case which I consider yet not very rare, does this setup actually make sense if you want high performance?

An SDDC solution would have some sort of scheduler, and your IaC CPUs are virtual CPUs on real CPUs, so in most pessimistic scenarios your Spark shares resources with say corporate mail server or some backup solution. And there can be priorities set between them, so the sharing is not equal. And collision cases described below from random towards a notorious DOS (Denial of Service) bias.

Collision case 1. Then your Docker Swarm, or Kubernetes scheduler would "see" a machine and say, well, the load there was not so much, so I take it - and then it finds what if was not expecting because the SDDC system scheduler has decided to take it already for something else.

Collision case 2. Just in the moment the IaC scheduer decides to compute something, the SDDC scheduler decides to unschedule it, so what would happen is kind of paralyzing spikes setting IaC to a slow motion mode.

Answer 1

I disagree with your definition, kubernetes, openshift and other are scheduler and as such should go in the SDDC part. IaC things are Vagrant/Terraform/Cloudformation/Vmware Cloud management. They are about writing code to define the infrastructure.

With this in mind k8s may fit both, a deployment.yml is IaC to drive the k8s SDDC.

Let take exemple for a k8s cluster running on top of vSphere infrastructure:
You have a scheduler k8s on top of another scheduler vSphere. SDDC on top of another SDDC.

K8s is concerned by its node load (which are vSphere VM), vSphere is concerned by its nodes load (ESXi hosts).
With this in mind k8s will move pods (containers) around its nodes, where vSphere will move k8s nodes around its hosts.

This indeed could lead to a spike or freeze of k8s nodes when both wish to move their loads at the same time. The solution is just an architecture solution: disable vSphere DRS on your k8s nodes so there won't be a move of your k8s nodes around while they are already moving containers.

Each time you overlay systems with the same behavior, you have to plan their definition properly to avoid race conditions.

Worst case would be running a docker swarm cluster on top of kubernetes on top of mesos on top of vSphere infrastructure on top of Nutanix partitions. (Absolutely doable but I don't think anyone would do that out of demonstration purposes)

So yes, this setup makes sense, it mainly require a careful design of all layers to achieve the best performances.