Local optimizations compose badly
Identical mechanisms at adjacent layers do not add up. They interfere.
What it is
The general lesson: identical mechanisms at adjacent layers do not add up. They interfere. The lower layer's batching is defeated by the upper layer's batching. Expect the same with caches, retries and batching.
From Physical Substrate & Performance Laws › Disk and flash
Layered optimizations compose badly. Nagle plus delayed ACK is the standard case, but the pattern is general: buffering at two layers, framing disagreement between two hops, compression at two levels. When you add a mechanism at one layer, ask what the layer below already does.
From Data Movement › Zero-copy, Nagle, TLS
Timeout budgets show the general shape, and it recurs: timeout ordering is a global invariant, and each team can edit only a local configuration. No amount of local correctness produces it, because correctness is a property of the composition and nobody owns the composition. This is the same structure as the retry-budget problem: each tier's retry count is safe alone and lethal composed. The only fixes are:
From Emergent Failure Dynamics › Timeout cascade
Where it appears
In 7 mechanisms. Each says what the mechanism is about, then how this principle shows up there, in its own words.
- Data Movement
How do records cross between two systems that have independent rates, independent availability and independent lifetimes, and what are you willing to lose when the assumption underneath the crossing breaks?
Local optimizations compose into global pathology.
Read it inHow flow-control mechanisms composeZero-copy, Nagle and TLSData movement, stepped back
- Distributed Systems Coordination
Who is allowed to decide, and how does everyone else learn the decision without asking?
The clients are the largest uncoordinated population in the system, and their combined behaviour is emergent, not designed.
Read it inThe actor is no longer a memberRebalancing: a controller with a loop delay measured in hours
- Fault Tolerance & Delivery Semantics
What is the cheapest thing that can be safely done twice?
A per-hop policy becomes an exponential policy in a layered system.
- Emergent Failure Dynamics
What happens when nothing is broken, every component is healthy, and the system's own response to stress makes the stress worse?
Here is the general shape, and it recurs: timeout ordering is a global invariant, and each team can edit only a local configuration.
Read it inTimeout cascadeAutoscaling oscillationRetry amplificationThe loop as an objectWhat overload defences forbidReview a design for runaway failure
- Table & Storage Maintenance
Who pays for the gap between the order data arrives in and the order it is read in: the writer, every reader, or a background job, and what is the break-even?
The target selection must be jointly size-aware and key-aware.
Read it inWhen packing ruins clustering, and defragmentation
- Schema Management & Evolution
What happens when the shape of your data changes on its own schedule, and nobody can change the readers, the writers and the old data at the same moment?
Explicit column lists stop the propagation, but at the price of a manual edit per hop for every addition.
Read it inPropagation through a derived DAG
- Measurement & Experimental Method
What would have to be true for this number to mean what I think it means?
Closed-loop lets the system choose its own arrival rate.
Read it inThe harness is not neutral: open versus closed loop
Related
Made on the same pages most often.