A cap replaces an estimate
Where a value cannot be predicted, systems substitute a cap.
What it is
A runaway-query killer is what a system does when it cannot estimate: it stops predicting and starts truncating. A query's cost distribution is heavy-tailed: a small fraction of queries consume most of the resource. So the mean is set by the tail, and a hard cap moves the mean far more than any tuning of the median case. The cap does not need to be accurate. It only needs to be finite.
From Scheduling & Workload Management › Runaway killers
Under occupancy pricing, then, a governor can only observe and cap. On a heavy-tailed cost distribution a fixed cap captures most of the savings without a model, and it should trip on early evidence, because a kill at 95% pays the full cost and returns nothing (runaway killers). Where the meter allows it, estimate and reject at admission instead.
From Cost Mechanics › Governors
A watermark is a declared lower bound on the event times of future arrivals.
From Time & Temporal Semantics › Watermarks
Where it appears
In 3 mechanisms. Each says what the mechanism is about, then how this principle shows up there, in its own words.
- Core Data Structures
How do we choose a data structure when the real cost is set by the hardware, not by Big-O?
A watermark is the streaming replacement for a sort, and its price is exactly the assumption it makes plus the cost when the assumption breaks.
Read it inWhat comes next is a relational question
- Scheduling & Workload Management
Which work runs now, which work waits, and who pays for the waiting?
> When you cannot predict a cost you cap it, and the cap should trip on early evidence rather than on elapsed time.
Read it inRunaway killers and result governors
- Time & Temporal Semantics
When am I allowed to act on data that may still be incomplete, and what do I owe the reader if I was wrong?
A watermark is a declared lower bound on the event times of future arrivals.
Read it inWatermarks: declaring completeness
Related
Made on the same pages most often.