What functional differences are there between kubernetes, rancher and mesos?

Question

What are the main feature differences the popular container orchestration tools kubernetes, rancher, and mesos?

What would lead you to choose one over the other?

Answer 1

First of all, Rancher actually contains implementations of both Kubernetes and Mesos within itself. However, they did make their own system called Cattle which is heavily based on Docker's Swarm. I'll touch upon this in the section for Rancher.

Secondly, since each one offers similar base features (load balancing, resource isolation, etc) I'll try to go more into what makes them different rather than focusing on differences among those common features unless they're significant.

Kubernetes

Highly performance focused, also featuring cloud storage orchestration (a feature missing from Mesos, although there's probably a plugin for it). Has API options to allow for automated scaling of resource usage as needed by individual containers (and for the reverse if a container is not being hit hard while others are).

Something important about Kubernetes is that unlike other container orchestration software it doesn't provide a comprehensive configuration or any kind of comprehensive self-healing. Instead, it focuses on continuous deployment of multiple apps with an easy rollback system at the app level (as a result you might want to look into micro-services when using it).

Each app is a small piece and can be deployed/configured individually. They make a point in their docs to say that Kubernetes is not a traditional PaaS (platform as a service) system since it lacks middleware for virtual hardware or databases and it doesn't build your app itself. It's designed to (as they say themselves) remove the need for manual container orchestration, instead automating the process by continuously pressing towards a target app state.

Mesos

Monolithic in comparison to Kubernetes. Focuses on the big picture moreso than the individual services, although it still allows for management of individual services. Provides built-in middleware for things like Spark, Hadoop, etc. Best usage of Mesos will involve many plugins as it's designed to be easily extended.

If you need fine-grained control over managing your application (insofar as there's a plugin available for what you want to do or you have a team member willing to build one if there isn't) you'll want to use Mesos.

Rancher (and Cattle)

Potentially the best option in that it is itself a superset of the previous two, having an implementation of both. This might also be seen as a downside as more complication in your management software is rarely a good thing and might lead to unforeseen issues.

Rancher features an application catalog that allows for one-click deployment, something Kubernetes doesn't have due to its design philosophy. However, seeing as Rancher has an implementation of Kubernetes, you can use Rancher if you feel these features are missing from Kubernetes.

Cattle is based off of a stack system, where you group related services together. It also houses Rancher Compose, a feature similar to the Docker service of the same name. This is probably the most interesting part of Cattle, the rest of it being fairly standard (although the Secret management is still in beta). If you have experience with Docker Compose you should read up on it here (I don't, so I'm probably not the best person to write about it).

Resources: "What is Kubernetes?", "Overview of Rancher", "mesos.apache.org: 'What is Mesos?'"