Graceful degradation, the honest version

In what sense is spilling graceful, and why does the system collapse on the spill device?

"Graceful degradation": the honest version

The phrase "graceful degradation" is misleading, and it is this mechanism's most repeated claim, so we close it out precisely.

Degradation here is graceful in the asymptotic sense: the operator completes instead of failing. It is catastrophic in the latency sense, for three independent reasons:

1. the bandwidth ratio (What spilling actually costs);
2. the pipeline break, which makes latency non-proportional to the spilled fraction (What spilling actually costs);
3. skew, which can prevent convergence entirely (Hash spill, and the arithmetic of a single pass).

Both statements are true at once. Confusing them is the error. "It will finish" and "it will finish on time" are different guarantees, and spill gives only the first.

There is a fourth reason that appears only under load. Rationing rationed memory carefully. Nobody rationed spill bandwidth. When memory pressure is correlated across queries (which is the normal case, since whatever raises one query's footprint usually raises many), every query spills at once, onto a local disk with no admission gate. The escape hatch becomes the contended resource, and its queueing response is the same 1/(1−ρ) curve as everything else.

This is the sharpest self-inflicted wound of this mechanism. You build an elaborate admission system for memory precisely so that overload turns into spill. Then overload turns into spill simultaneously, everywhere, onto an ungoverned device. (This is the third incident.)

One-line compression: spill guarantees completion, not latency, and under correlated pressure the escape hatch is the least governed resource in the system.