Spot capacity: the vendor's idle stock, resold

What decides whether spot is cheap, and why does a gang job almost never survive on it?

Spot capacity is the vendor's own idle inventory resold at marginal cost. That is why the discount is so large. It is also why availability is correlated with exactly the wrong thing: spot disappears when total demand peaks, which is when you most want it. The cheapness comes from sharing in someone else's multiplexing, and the correlated risk comes with it.

Restart granularity decides the economics

and the work lost is proportional to the checkpoint interval (a checkpoint saves the job's progress so a restart can continue from it).

• Checkpoint every 5 minutes: you lose 2.5 minutes on average per interruption (half the interval).
• A 6-hour job that restarts from scratch: you lose 3 hours on average per interruption.

Checkpointing, and recomputing lost pieces from a record of how they were made (lineage), therefore have a directly computable dollar value. They are the only thing between an 80% discount and an unusable platform.

The gang effect

This arithmetic kills naive spot use. With n nodes that each face an interruption probability p per hour, a job that needs all nodes survives one hour with probability (1 − p)ⁿ.

The work is AND-coupled: all parts must survive. Failure probability concentrates in a way that is easy to underestimate. The standard resolution, which everyone reaches, is an explicit decision about which parts are recoverable: the coordinator and the state go on on-demand machines, the workers go on spot.