Congestion collapse

How can throughput fall as offered load rises, with every health metric green?

Congestion collapse needs its own name, not just "overload", for this reason: throughput decreases as offered load increases. Ordinary overload plateaus: latency climbs and completions stay flat. Collapse needs wasted work to consume the same capacity as useful work.

The derivation

Let the queueing delay be D and the client timeout be T. When D > T, every item the server finishes has already been abandoned by its client. Look at the server's view:

• Utilization is 100%.
• Per-item service time is normal.
• All health metrics are green.
• Goodput is zero.

The gap between throughput and goodput is the whole phenomenon. The server cannot measure it, because the information about whether anyone still wants the answer exists only at the client. Who pays when the assumption breaks? The server assumes its output is wanted. The client pays: it receives nothing while being charged for everything.

The historical evidence

The 1986 collapse of NSFNET (the main US academic network of that time) is the classic case. Useful capacity fell by roughly three orders of magnitude, from 32 kbps to 40 bps, with no hardware fault, because the network was busy carrying retransmissions of packets that were already in flight. The fix was not more bandwidth. It was AIMD (additive increase, multiplicative decrease) at the sender: raise your sending rate slowly while things go well, and cut it by a fixed fraction (for example in half) as soon as loss appears. It is the only control law fast enough to unwind an exponential, because a fixed-size decrease cannot keep up with multiplicative growth. (This is the principle that a fixed rate meets exponential growth and loses.)

The modern fix at a service is the same idea in a different place: refuse the excess cheaply at the edge, so the capacity goes to work someone is still waiting for (load shedding). A system without shedding does not have a higher capacity. It has a lower one, which appears exactly when you care.