The system is its own client

Why does deferred work become a debt whose interest rises with the balance?

CaseConstructed case · Selin at Wayhop

Flat traffic, and a p99 that climbs all afternoon

A few weeks later Selin sees something that has no villain. The search service (the nodes that answer "which drivers are near me?") runs on the JVM (the Java virtual machine, which manages memory with a garbage collector). Its p99 latency (the latency that 99% of requests beat) slowly climbs over an afternoon. Traffic is flat. Nobody deployed anything. The heap fills, garbage-collection pauses grow, requests pile up, memory use grows, and the pauses get worse.

Three more things happen in the same month:

The key-value store behind pricing falls behind on its background merging ("compaction"). Read latency doubles at constant traffic.
The autoscaler adds servers, then removes them, then adds them again, every few minutes. The graph looks like a saw.
After a 10-minute outage, the queued work replays and the system falls over a second time. When it recovers, its caches are empty, and a third collapse follows.

Bruno asks: "Which client misbehaved this time?" None did. This article explains why.

How it ends →

Every earlier article needed something from outside: clients that retry, a trigger that pushes load over a threshold, a dependency graph, a degraded disk. This article needs nothing from outside. Remove every misbehaving client, fix every correlated dependency, give the failure detector perfect information, and a single process serving well-behaved traffic at a constant rate can still destroy itself.

The constraint is that systems defer work, and the deferred work must eventually be done with the same resource that serves foreground (user-facing) work. So the system is its own client:

Allocating memory defers cleanup to the garbage collector.
Ingesting writes defers merging to compaction.
A cache miss defers its cost to whoever warms the cache.
An outage defers processing to a catch-up phase.

In each case a debt builds up, and paying the debt competes with the activity that creates it.

That alone would only be a queue: unpleasant, but linear and limited by simple arithmetic. What turns it into a spiral is the second property: the cost of paying the debt rises with the size of the debt. That is the whole article in one sentence.

Deferral is a loan against a resource you will still need, and the interest rate rises with the balance.

A plain queue drains linearly and is merely slow. These do not: GC cost per reclaimed byte rises with occupancy, compaction cost per write rises with overlap, cache-miss cost rises with churn, and recovery cost rises through downstream failure. So deferred work accrues interest, and every deferral needs a hard stop, not a best-effort background task. Predictive form: for any "we will do that later" mechanism, ask whether the later cost is constant or increasing in the amount deferred. If it increases, the system is bistable and needs an explicit bound. The bound will look like a deliberate outage, which is why it is usually left out.