Load stops being an independent variable

What changes when the offered load depends on the system's own latency?

CaseConstructed case · Selin at Wayhop

A 3-second stall, a 40-minute outage

It is 18:05, the start of the evening peak. A routine index rebuild makes the pricing database stall for 3 seconds. Nothing else happens. No disk fails. No server crashes. Traffic is the normal evening traffic. Yet within a minute the pricing database is receiving roughly 200 times its normal load, the booking service stops answering, and riders see errors for 40 minutes.

Bruno asks two questions. "Why did a 3-second blip cost us 40 minutes?" And, when Selin proposes more servers: "Why not just add 50% capacity?" She will find that neither question has a simple answer, and that the same pattern shows up in four more small puzzles: a spike at exactly midnight, a cache restart that crushes the database, a "smart" router that makes things worse, and a queue that makes every request late.

How it ends →

Most performance reasoning treats load as something that comes from outside. You know λ, you compute the response time for that λ, and you decide whether you have enough servers. Fault-tolerance reasoning treats faults as something that comes from outside too: a disk dies, a node dies, a data center loses power. The cause is external, and the system's job is to absorb it.

This mechanism is about a different situation: λ becomes a function of the system's own state. The offered load depends on the latency clients observe (slow replies cause retries). The available capacity depends on the offered load (heavy load makes servers slower). When those two dependencies close into a circle, you no longer have a queueing system with a fixed capacity. You have a control system with a gain.

Two things change.

The constraint that appears is positive feedback with delay. A change gets amplified, and the amplified effect arrives back at the input some time later.
The constraint that stops mattering is the external fault. Nothing has to break. Every component can be healthy, every disk intact and every line of code correct, and the system can still sit in a stable, broken state for hours.

The governing question moves from "does the system survive the fault?" to "does the system's response to stress reduce the stress or increase it?" In other words: is the loop gain above one or below one?

This also changes how we think about independence. Queueing theory assumes that the arrival process (clients asking) is independent of the service process (servers answering). Retries, timeouts, caches, health checks and autoscalers each exist precisely to couple the two. The principle that independence is assumed everywhere and true almost nowhere stops being a small correction and becomes the whole subject.

Every mechanism that protects a single request from failure is a mechanism by which one request's failure recruits another request's capacity.

That is the tension of this mechanism. Each component behaves in a way that is sensible locally. The aggregate is unstable.

Scope of this mechanism

This mechanism studies the dynamics of mechanisms, not the mechanisms themselves. Queueing arithmetic, the Universal Scalability Law and tail-at-scale arithmetic are used here as tools, and explained where needed. How retries, idempotency, replay and failover must be implemented to be correct is a different topic (making a repeat harmless); here we only study what they do to load. How a cache is filled and invalidated is also a different topic (stampede); here we study only the stampede dynamic. Admission control and quotas as business policy (admission and queuing), and backpressure and rate limiting as protocol features (flow control), are used here only as stability devices.

What kind of knowledge this is

The core results are permanent. Queueing nonlinearity as ρ → 1 and the arithmetic of branching processes (a retry multiplier above 1 diverges) are mathematics, not engineering fashion. The specific pathologies are current engineering reality: GC death spirals are a product of managed runtimes, and cold-cache collapse is a product of one particular price ratio between memory and storage. "Metastability" was named only recently (a 2021 research paper at HotOS, a systems workshop), although the phenomenon is old, and the boundary between "metastable" and "just slow to recover" is genuinely unsettled. Treat the vocabulary as newer than the phenomena.