Cloud environments in AWS allow for multi-tenancy managed by the user himself, classic example are container orchestrators such as ECS or Kubernetes.

When you have two services, one needs memory another cpu and you put these in a single cluster. Then scaling-up is relatively trivial. Each time you need more capacity in terms of either cpu or memory, add more capacity. Since EC2 capacity means units in cpu and memory both.

Scaling up based on a single metric can very easily achieved using CloudWatch Alarms.

When scaling down, in order to reduce cost it requires to take into account both memory and cpu limits and not let any of the two drop below the required amount.

Since unfortunately CloudWatch Alarms do not allow to use boolean logic or take into account multiple metrics.

What is a good way to implement scale down of capacity for an auto scaling group?

  • Can you please clarify whether your question refers to scaling ASGs in general or specifically scaling Docker instances? – user2640621 Oct 29 '17 at 23:35
  • When I write “Auto Scaling Group” I mean the EC2 service called ASG, not Docker containers. – Evgeny Oct 30 '17 at 16:51
  • It's still not clear whether you're referring to scaling EC2 instances that host Docker containers specifically, or EC2 instances in general including traditionally hosted applications that don't use Docker. – user2640621 Oct 30 '17 at 20:57
  • It doesn’t really matter, as long as there is some multi-tenancy in place. For example a Mesos cluster running non-Docker framework is also applicable in this question. – Evgeny Nov 1 '17 at 9:27

Autoscaling is a good case for machine learning

This is a hard problem to do well.

What you really want is something like Nest Thermostat for your EC2 infrastructure.

There are (aforementioned) multiple dimensions of resource demand/limitations.

  • CPU
  • memory
  • disk space
  • disk IO
  • network IO
  • concurrency/latency/queue depth

There are multiple indicators of demand (in addition to the above).

  • concurrent unique visits/sessions
  • pages per visit/session
  • rate of engagement/interaction feature usage

There are multiple common patterns of changing demand over time.

  • daily user demand cycles
  • weekly user demand cycles
  • monthly ...
  • annual ...
  • special event/days
  • DDoS load
  • media/marketing exposure traffic spikes

There are multiple financial decision factors.

  • does revenue scale with traffic? How? (How conservative do you need to be?)
  • is there a hidden cost to control (transaction costs, limits)?
  • what's the cost model of scaling? (things in a pipeline scale together, things in a load-balanced cluster scale independently)

Before long, if you try to hand-optimize on hand-selected features you're going to have a monster of technical debt that is possibly more complicated than any other logic in your site. Amazon makes more money when you err (with a large margin) on the side of caution, so their tools will probably never get close to what you want.

Instead, choose an architecture/technology stack that can grow/scale so you don't have to get it exactly right the first time. Then pick a few factors which you think are obvious. Then try to come up with a way to sort multiple representative possibilities in order of preference. Then collect some real world data covering all those points. If you're lucky, a simple obvious hand-coded solution will jump out at you from looking at the data. If not, code up something that will give you an approximate model f(x1,x2,x3,x4) --> y * app nodes, using an appropriate algorithm.

I bet you didn't think this one was going to be so much fun!

  • I don't think this answer is appropriate. It's not clear from the title (and perhaps it should be edited) but the question specifically asks about scaling Docker instances. There are only two resources in contention on an instance that hosts Docker containers - allocated CPU, and allocated memory. All other resources (including all the ones you've listed) are in contention on the containers. Your answer is valid for EC2-based applications, but doesn't apply to Docker. – user2640621 Oct 27 '17 at 22:37
  • I disagree with your interpretation of the OP question. Docker is one example, but the author seems to want help coming up with a broad and general solution, which renders scale targets in EC2 units. – Jeremy Oct 29 '17 at 20:01

To be able to scale in with confidence on ECS, there's a few strategies:

  • Create a 3rd custom metric which tracks the greater of the two metrics. For example, if CPU allocation is at 60% and memory allocation is at 70%, the metric should be set to 70%.
  • Choose one of the two resources (CPU or memory), and always allocate a higher percentage of it to each container compared to the other resource. This way, you will always reach a shortage of your preferred resource before the other, and you don't have to worry about scaling up or down on it. While this is the easiest solution, the obvious downside is that you can't optimise for services that may be CPU or memory intensive, and you may end up with wasted resources.
  • Do autoscaling without using CloudWatch metrics. Tools such as https://github.com/ameir/ECSpander are available to assist with this.

Kubernetes doesn't use CloudWatch metrics for scaling, it manages scaling by setting the ASG desired number of instances through internal mechanisms and therefore isn't subject to the issue you've described.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.