0

Let's say we have a continuous process where each two hours we receive some data which has to be analysed by a fleet of workers on AWS EC2 instances which always complete their job within two hours (let's say normally within 90 minutes).

What would be the most cost-effective solution in this case?

  1. Use Spot blocks with finite duration of 2 hours?

  2. Use fleet of reserved EC2 instances?

Though spot blocks seem to be less expensive, I am concerned about their availability as data has to be processed immediately after receiving within the next 2 hours.

1

Anything that "needs" to happen in a given time frame is not a candidate for spot instances, including spot blocks.

Spot capacity is, by definition, surplus/idle capacity, subject to availability, so your spot block may simply fail to start, as capacity may not be available. It is less expensive for exactly that reason -- AWS is selling surplus capacity for which no one is currently willing to pay more.

Information about the pool of compute capacity that is currently flagged internally as eligible for participation in the spot market is not publicly accessible, and although it was possible to make some reasonable assumptions in the past about capacity based on signals you could obtain from the real-time pricing, the spot pricing algorithm was modified in 2017 in a way that largely masks those signals (and the change, in my opinion, was a good thing for any customer who was not aggressively gaming the system).

Spot blocks are intended to run for a committed period of time, so they are less likely to be interrupted than other spot instances, but they can initially start up successfully and then subsequently be interrupted at any time.

In rare situations, Spot blocks may be interrupted due to Amazon EC2 capacity needs. In these cases, we provide a two-minute warning before we terminate an instance, and you are not charged for the terminated instances even if you used them.

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-requests.html#fixed-duration-spot-instances

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.