DevOps Stack Exchange is a question and answer site for software engineers working on automated testing, continuous delivery, service integration and monitoring, and building SDLC infrastructure. It only takes a minute to sign up.
Sign up to join this community
While trying to create spot instances I notices that the Price for almost all instance types (from m3 , c3, c4) are always higher than the On-Demand price of corresponding variants in us-east-1c and us-east-1f regions (for almost 3 months)
Can someone explain why? Is AWS falling short of VMs in those AZs?
You notice that you now have access to 6 availability zones in us-east-1, where you previously only had 4 or 5.
Adding capacity is the reason that new availability zone is now available.
Because of the way the spot market works, there is no way to know what the highest winning bid was, at any given time... because the current market price is based on the lowest winning big for each round. That sounds counter-intuitive, but here's the logic: given n available instances in the spot market, the system identifies the highest n bids. These bids are the winning bids. The lowest of these winning bids is the price all of the winners will pay, and becomes the new current market price.
Each time the market shifts (new bids arrive, or active apot requests are canceled, or on-demand instances are started/stopped, presumably resulting in the available spot pool growing or shrinking), the system re-evaluates all the bids. If you are no longer a winning bidder, your instance will be terminated, but if you're a new winner, you'll get the instance you've been waiting for.
Based on what's publicly known, we can conclude these things:
Therefore, when the spot price is at maximum, there are no unsold spot instances. We can logically conclude that this is necessarily true, because:
a single winning bid that is lower than the maximum would cause the market price to be lower than thr maximum; therefore, it defies logic that there could be any spare capacity, because this implies that all current winning spot instance bidders made outrageous bids when they could have obtained the instances for substantially less money either by placing a smaller spot bid or by using an on-demand instance
it also defies logic that AWS would artificially inflate the price above the on-demand price, because there is no rational reason for someone to pay more in the spot market when the could use an on-demand instance.
We can further conclude that one of two things has happened to cause this, both of which have the same net result:
NULL because there is no price at which you can obtain a spot instance. Or, you might say the current market price is infinite, because regardless of how high you bid, you can't have a machine that doesn't exist. Some might naïvely say the current price is $0 but that is definitely incorrect and would send the wrong market signal. Where this leaves us is that the most logical representation for the price in a zero-capacity condition would be the maximum/ceiling price. This gives a clear market signal to any sensible algorithm -- don't bid in this market, the value is not there; orWhich of these two conditions is actually true doesn't matter, because the net impact is the same -- no available capacity.
If that second possibility doesn't make sense to you, that's because you are a rational person. There are always people who try to game the system.
Instances in the spot market are ephemeral, since the market shifts are outside your control. I've had spot instances run uninterrupted for many months on end, and I've had them cycle up and down many times over the course of one day. This is part of the market.
It is possible to improve the chance of a spot instance persisting by placing a high bid... but it is not possible to guarantee that a spot instance will keep running, regardless of how high you bid... because even if all the high bidders colluded with each other and bid the exact same high price, the capacity is finite, so if all current spot instance capacity is sold, not all of the high bidders would have machines allocated to them.
However, there's always that guy who thinks he's smarter than the rest of us, bidding far too high, presumably on the assumption that his instances are likely to continue running, and that since price spikes are transient, he expects only to pay the high price for only a short time. If there are such gamblers in the market, then they would have the same effect of pushing the price to its limit when capacity is constrained. This becomes clearest when you consider the fact that if there were only 1 available machine, then only 1 person needs to place an outrageous bid, to set the market price unreasonably high.
The maximum spot price bid is now 10x the on-demand price. This has not always been the case. The logical conclusion is that there are indeed those who bid outrageously high, because they are trying to game the system and use spot instances for a persistent workload, so they bid high to try to keep those machines running. It's a flawed strategy, but it does explain the prices hitting the ceiling and sticking there when capacity is limited.
Essentially yes. From what I've observed when there is limited capacity in that availability zone the spot price jumps to 10x the on demand price.
Be sure to use as many AZs as possible (6 currently in us-east-1) when using spot instances to provide better guarantees you can launch an instance.
