Physical Substrate & Performance Laws
Why do trade-offs exist at all?
Who this mechanism is for: a developer who writes code, calls APIs and knows basic SQL, but has never studied how a disk, a CPU cache or a network really behaves. You do not need any database knowledge. Every technical term is explained the first time it appears, and again in the glossary at the end.
The story we will follow
Meera is a backend engineer at Streamly, a small company that sells a dashboard product. Customers send click-events (who clicked what, and when). Streamly stores these events and lets customers ask questions such as "how many users from India clicked Buy last week?"
Streamly launches next month. During the launch period Meera will meet five problems, one at the start of each article below. Each problem looks like a bug. Each one is actually a law of physics or mathematics. At the end of each article, we go back and solve her problem.
This is the main idea of this whole mechanism:
How this mechanism is organized
- 1
Seven root causes
Why does every storage system have trade-offs?
CaseWhy not one storage that does everything?
- 2
Storage media
What are disks, SSDs, tape and cloud storage made of, and what does that force on us?
CaseThe slow nightly job and the 12,500-IOPS SSD
- 3
CPU and memory
Why does it matter more *how predictable* your code is than *how much work* it does?
CaseThe in-memory scan that is still slow
- 4
The network
Why is a network different from a disk, and why do networks collapse under load?
CaseA 1-second p99 at 100 servers
- 5
Performance laws
Which simple mathematics controls every queue, thread pool and fan-out?
CaseLaunch day: 80% CPU, then a 12-second p99
- 6
Why these limits last
How do we use all this to judge new technology?
After that: a summary, hands-on labs, interview questions and a glossary.
One request: do not skip the numbers. This mechanism quotes many figures ("80 µs", "64 bytes"). The ratios between them are the real content.
Can I see it myself?
Write amplification: same-age vs. mixed-age writes
A flash translation layer simulator filled to 50–90% and overwritten with same-age (whole files) and mixed-age (random pages) writes.
SimulatePython with NumPyThe cost of utilization, u/(1-u)
A log-structured store with a greedy cleaner at 50% to 95% live pages: pages copied per page of new space, against u/(1 − u).
SimulatePythonSorted vs. shuffled branch benchmark
Sum the elements above a threshold over shuffled and sorted data, compiled at -O1, -O2, -O3 and with a branch the compiler cannot remove.
MeasureC compiler, PythonPut your own code on the roofline
Measure this machine's memory and compute ceilings, then place a scan, a filter and a hash aggregation on its roofline.
MeasurePython with NumPyReproduce incast on a laptop
One receiver asks N senders for 32 KB at once, N from 1 to 500, with synchronized and jittered replies: p50, p99, p999 on loopback.
MeasurePythonFit a USL curve to something real
N processes each scanning their own array (memory-bandwidth-bound): fit the Universal Scalability Law at N ≤ 8 and test the prediction beyond.
MeasurePython with NumPy, SciPy
Can I reason about it under questioning?
Interview questions
Short questions on disks, memory, networks and performance laws that an interviewer might ask, each with an answer you could give aloud and a link to the full explanation.
Scenarios
Longer problems on hardware limits and performance laws, of the kind asked in senior interviews: each one estimated with numbers, diagnosed, and answered with its trade-offs.
The tensions this mechanism takes a position on. Each recurs across mechanisms.
Space vs. time
For example: compression, indexing, materialization, caching.
Read-optimized vs. write-optimized
For example: B-tree vs. LSM, merge-on-read vs. copy-on-write.
Fail-fast vs. fail-degraded
Reject or continue when something is wrong.
Latency vs. throughput
For example: batching, group commit, micro-batch.
Consistency vs. availability vs. latency
For example: quorums, replication modes.
Ideas this mechanism meets that keep coming back, mechanism after mechanism.
Variance costs more than volume
Variance costs more than volume.
Costs are conserved
Costs are conserved, never removed.
Immutable storage grows indirection
An immutable substrate grows an indirection layer plus a collector.
Granularity mismatch
Granularity mismatch creates amplification.
Fixed cost per operation
Fixed cost per operation implies a natural batch size.