We're running like 100+ servers on AWS EC2 with like 20 microservices all load balanced and autoscaled with 2 to 3 instances.
The cost has become....very significant.
I'm wondering if migrating these to Kubernetes or ECS would be cheaper?
DevOps Stack Exchange is a question and answer site for software engineers working on automated testing, continuous delivery, service integration and monitoring, and building SDLC infrastructure. It only takes a minute to sign up.Sign up to join this community
From your comment I assume a cpu load under 5% in average considering all instances together.
Each of your actual instance can be a pod on a kubernetes node, using quite larger instances you can reduce to a dozen of m3 or m4 machines with a single (or two) load balancer in front for the same quality of service.
You'll have machines used at 70% (ideally) and using an autoscaling group with a scaling policy to take care of spinning up a new node in case of burst.
The main idea is that you should be able to host on fewer machines with a better average load and thus reduce your cost, the first net gain is the reduction in number of load balancers, but this could be debated with Application Load balancers which allow to route by context or host.
Be warned there's a learning curve with kubernetes, but according to your description this sounds the perfect use case.
Beware of the 'all in k8s' trap, some things are better handled out of kubernetes, like large database systems or CPU/memory intensive task as they disrupt the load repartition and brings some more complexity. I.e: keep both possibilities, k8S and usual ASG/instances based hosting.