Little's Law
Concurrency = Throughput × Latency.
What it is
Little's Law is a conservation identity: no assumptions about distributions or independence. If 100 customers per hour enter a shop and each stays 30 minutes, then on average 100 × 0.5 = 50 customers are inside. It cannot be wrong.
| Where | Form |
|---|---|
| NVMe queue depth | IOPS = QD ÷ latency |
| Memory-level parallelism | bandwidth = outstanding misses × 64 B ÷ 80 ns |
| Bandwidth-delay product | in-flight = bandwidth × RTT |
| Here | L = λW |
When something is slow and you cannot find the bottleneck, first solve for the missing term. Usually the concurrency is too low, which no average-based metric shows.
You can usually measure two of the three terms. Queue depth hard to measure? Compute L = λW.
Example (a limited pool). A database connection pool has 50 connections and each query holds one for 100 ms. Then λ_max = L / W = 50 / 0.1 = 500 queries per second. If traffic doubles to 1,000/s, W cannot halve, so requests wait, and the waiting time becomes part of W. A pool of 50, a thread pool of 200, a max_concurrent_requests setting: each is you choosing L. By choosing it, you chose the relation between throughput and latency. You cannot choose both.
From Physical Substrate & Performance Laws › Little's Law and queues
Little's Law connects latency and throughput. It told you how many requests to keep in flight on a fast SSD. Here it sizes your connection pool, your consumer concurrency, your credit window (how much a receiver allows the sender to send) and your in-flight request limit. Same algebra, different units.
From Data Movement › The epistemic limit
Where it appears
In 9 mechanisms. Each says what the mechanism is about, then how this principle shows up there, in its own words.
- Data Movement
How do records cross between two systems that have independent rates, independent availability and independent lifetimes, and what are you willing to lose when the assumption underneath the crossing breaks?
Little's Law connects latency and throughput.
- Storage: Physical Organization
What should sit next to what, given that I must decide at write time and find out at read time?
The minimum efficient request size and the minimum number of bytes in flight are the same number.
Read it inWhat physical organization forbids
- Transactions & Concurrency Control
What happens when many users change the same data at the same time, and how do we keep the result correct?
Remove the waiting and you remove the need for concurrency, and therefore the entire need for concurrency control.
Read it inDeterministic and partition-serial executionWhy concurrency is a different kind of problemChoose a concurrency scheme
- Consistency & Isolation Models
When many users, many objects and many copies exist at once, which histories does the system promise never to show, and who writes the code for the histories it does show?
Second, the window is almost always the cheaper number to attack: shrink W.
Read it inThe governing arithmetic: three resultsRead Committed and the lost update
- Memory & Resource Management
Who is allowed to hold how much memory, for how long, and what happens to the work when the answer is less than it asked for?
Spilling to relieve memory pressure increases the number of concurrently resident queries, which increases memory pressure.
Read it inWhat spilling actually costs
- Distributed Systems Coordination
Who is allowed to decide, and how does everyone else learn the decision without asking?
In short: RPC's third outcome manufactures idempotency and deadlines, and an absolute propagated deadline is admission control by Little's Law.
Read it inRPC: idempotency and deadlines
- Fault Tolerance & Delivery Semantics
What is the cheapest thing that can be safely done twice?
The design rule to remember: recovery should cost what was in flight, not what is stored.
Read it inWrite-ahead intent and the recovery scan
- Emergent Failure Dynamics
What happens when nothing is broken, every component is healthy, and the system's own response to stress makes the stress worse?
All escapes are the same move: bound the live set, not the heap.
Read it inGC death spiral
- Orchestration & Workflow
How do you make unreliable, unobservable, irreversible side effects behave like a pure function of a time interval?
A nested scheduling unit that waits on the pool it occupies will deadlock.
Read it inTask groups and reusable componentsSensors: polling versus push
Related
Made on the same pages most often.