I'm exploring doing kubernetes on a multicloud setup the configuration I want is a master connected to nodes in multiple cloud platforms. In my limited research I haven't found many tools to make this happen, from what I can tell rancher seems like the best option, but I don't think this fits within the traditional usecase. My question is does this sound like something rancher was made for? Honestly it seems more of a kubeadm usecase but the caveat is I want to use multiple managed solutions where multiple masters all share a common node pool.
(Rancher employee) Kubernetes itself is really not meant for that use-case. A cluster is generally a set of machines in the same provider with close proximity (latency) to each other.
- Etcd (and therefore API/CLI/UI/scheduling/everything) performance depends heavily on the (worst-case) latency between all members.
- Only one cloud-provider integration (storage providers, L4 load balancers) can be configured and most assume nodes are all in the same "region" or similar concept.
- Commonly used network plugins assume adjacency and/or provide no encryption or authentication suitable for communicating across untrusted networks/the internet.
- Communication between pods/services within the cluster has no consideration for where the "nearest" place to reach a pod for that service is, so latency from a pod to another service can be large (however far apart the two worst nodes are) and unpredictable (sometimes close, sometimes far). (There is an option for nodePorts to always go to a local pod, but if there is no local pod it will just get dropped).
You can make a "custom" cluster and add nodes from wherever and do what you're asking in Rancher, but you're not going to have a good time (no matter what product is building the cluster).