Why is the AWS EC2's spot price greater than the on-demand price?

Question

I was trying to provision spot instances via Ansible yesterday, and almost all my requests failed, even when I put my spot price == the on-demand price of that instance.

So, when I had a look at the spot pricing graph, I found something very interesting:

The spot price of the instance in us-east-1a is more than the on-demand price, which confused me. [in fact, ~5x times higher]

Aren't spot instances preferred for the low cost? If yes, then why is the price higher than the on-demand price?

According to AWS's docs:

Spot instances provide you with access to unused Amazon EC2 capacity at steep discounts relative to On-Demand prices.

Also, does this mean that people are bidding over the on-demand price? If yes, then why so? Aren't they better off with an on-demand instance?

Or did I understand the concept of spot instances wrong?

Answer 1

This is actually a great example of people slightly abusing spot. People are saying 'Our workload is really important but we don't want to pay full on demand price', so they set a bid price higher than on-demand on the assumption that it is very unlikely to be terminated, but still want to get the 'cheapest possible' spot price on offer.

There have been cases where people enter, for example, $1000 (I've been told of at least one time this happened) because they want the benefit of the spot market. Of course naturally at some point the demand comes in and the spot price SOARS to make people pay higher than on-demand.

The way the spot market works is that Amazon have X instances spare capacity, and they count from the top down until they fill the need for all X instances. The 'price', then, is the lowest price at which they can fulfill those X instances.

So imagine Amazon have 10,000 instances - well they will count down to (say) $0.43 until they've got those 10,000 instances fulfilled. But if that supply suddenly drops to 100 instances, then maybe a few people put bid prices of $10,000 for their 100 instances, suddenly they'll be paying that $10k per hour.

Tl;dr Understand how spot works, and set a cap you are prepared to pay.

Answer 2

There are 2 reasons for this:

• A lot of users are using the spot instance at some times (Think about batch processing, boot up 100 machines as spot instance and crunch away).

• For a spot instance you don't pay the bidding price, you pay the current spot price. The bidding price is the cutoff point. If the current spot price exceeds that of the bidding price AWS will terminate that instance.

This last on is also the reason that some user will massively overbid on the spot prices. They don't want their instance to shutdown every so often, so they bid a price so high that the spot price will never reach. Since they will only pay the current spot price the instance will be much cheaper 99% of the time.

Answer 3

A useful bit of information for understanding why someone would bid over the on-demand price can be found in the Introduction to Spot Instances:

Spot Instances can be used to help you meet occasional needs for large amounts of compute capacity (note that the default limit for Spot Instances is 100 versus the default limit of 20 for On-Demand Instances.) If your needs are urgent, you can specify a high maximum price (possibly even higher than the On-Demand price), which will raise your request’s relative priority and allow you to gain access to as much immediate capacity as possible given other requests and the Spot Instance capacity available at the time.

If you have a huge spike in demand for your service (perhaps being linked to from another popular site—see the Slashdot effect), bidding above the on-demand price for a spot instance would help you to get access to far more instances, as noted by the document.

Of course, this isn't sustainable in the long run, and you'd get a much better deal for a long-term compute by just buying on-demand instances (plus there's less risk of being outbid!).

If you're in a situation where you need a lot of computing power, fast—more than you could get from just buying on-demand instances—overbidding might make sense.

Answer 4

If you look at the charts closely, you will see that a spike is always of a very short duration - just enough time for the automated monitoring systems written by the owner to terminate those systems gracefully. In addition, you will occasionally find that the price drops down to 0 immediately following a spike. That is because all systems in that data center are in use as on demand systems, with no systems available for spot pricing the price is effectively zero.

When your spot instance flagged for termination, a message indicating this will be available on the system at the local meta-data uri of http://169.254.169.254/latest/meta-data/spot/termination-time. There will be 3 minutes until it terminates. More then enough time in most cases to handle termination automatically. Bidding above the demand price is only necessary for deployments which require more then a few minutes to terminate gracefully.

If it is not possible to design your system to terminate gracefully, archive data, etc in 3 minutes you can place a bid higher then the demand price to gain time. The system can even be designed to proactively monitor the current spot price and swap over before the price goes over. But for those times it does, you will need to make a business decision on how much it is worth for the time to terminate gracefully.

It is foolish to pay $100/hour for 4-5 hours in order to keep your system. However, if it will take your system 30 minutes to terminate all the processes gracefully, you can make a business decision how much it is worth to potentially lose any data, or degrade your horizontally scaled service. An e-commerce site with a net profit of $10,000 an hour can certainly afford to pay $1000 to keep 2 spot instances running for 15 to 30 minutes while bringing up demand systems and archiving data.

Web based application can use Elastic Load Balancer to help in addressing termination automatically. A smart implementor would put in place a set of scripts to handle the alert. They could maintain 2 low cost on demand instances which are load balanced - and then use up to half a dozen medium cost systems via spot instances to maintain high performance and spend less then a single on demand system of the same capacity.

Leave 3 of them paying up to $100/hour and 3 of them paying only up to half the on demand price. As AWS terminates instances, ELB will adjust automatically. Giving the automated system up to an hour to adjust for a mere $200.