Why fail-slow is worse than fail-stop
Why does a slow node hurt every fan-out request while a dead one hurts none?
Why fail-slow is strictly worse than fail-stop
The asymmetry is quantitative and severe.
A dead node in a fleet of N contributes zero work, is detected, is removed, and its damage is bounded: capacity drops by 1/N for the time it takes to replace it. The system's own machinery handles it. This is the case everyone designs for.
A node at 1/100 speed passes its health check, stays in rotation and continues to receive its 1/N share of traffic. It will complete that traffic, eventually, while holding caller threads, connections and deadlines for a hundred times as long. It consumes upstream resources in proportion to its slowness, not its throughput. So the damage inverts: the slower it gets, the more of the system's resources it occupies. The arithmetic of the resource-exhaustion cascade (resource exhaustion) applies directly, with L = λ × W and W a hundred times larger.
The comparison, with one bad node out of N:
| Fail-stop | Fail-slow | |
|---|---|---|
| Capacity contribution | 0 | about 0 |
| Detected | Yes | No |
| Removed | Yes | No |
| Caller resources held | None | proportional to its latency |
| Fraction of requests affected (full fan-out) | 0% | 100% |
| Duration | Until replacement | Until a human notices |
The last two rows are the point. Under full fan-out (every scatter-gather query, every distributed join, every quorum read) a single fail-stop node affects nothing, because it was removed, while a single fail-slow node is on the critical path of every request. One bad machine in a thousand degrades one thousandth of your fleet's capacity and one hundred percent of your responses.