Summary and key takeaways
The one ratio. amplification = bytes touched ÷ bytes needed. The denominator comes from the query. The numerator comes from the layout and the medium. A 200-byte row in a 4 KiB page costs 20×; in an object store with a 1-10 MB minimum sensible read it costs 10⁴-10⁵×.
The four decisions
| Decision | Question | Main result |
|---|---|---|
| Adjacency (Layout models) | Which values sit beside which? | Entity-adjacent (row) and attribute-adjacent (column) are exclusive. PAX is the bounded compromise and is forced on object storage. Columnar gains n/k times compression, but costs B/w on point reads. |
| Mutation (Mutation strategies) | What happens physically when a value changes? | An update is two facts. Copy-on-write costs U/d (unit size ÷ change size); merge-on-read moves cost to readers until a derived compaction interval τ*; delete vectors are cheapest where positions are stable; reclamation waits for the slowest reader. |
| Placement (Physical clustering) | Which unit does a record go to? | Clustering payoff ≈ min(G, 1/s). A composite key needs C_a ≤ U. Skip power is conserved across dimensions. Sortedness has rent. Metadata is data. |
| Substrate (Access and media) | What does one touch cost? | Characteristic size = latency × bandwidth. It sets every constant above. |
The principles of this mechanism, one line each
- Mismatch rule: mismatch between write, read and update units creates amplification. Read, write and space amplification are one phenomenon counted three ways.
- Batch-size rule: a fixed cost per operation gives a natural unit of
fixed cost ÷ per-byte cost; when the layer below gets faster, the fixed cost moves up and the design follows. - Little's rule: items in flight = throughput × latency. Together with the batch-size rule it is one number,
L × B: batching and pipelining are the only two ways to amortize a latency and cost the same buffer. - Conserved-cost rule: costs are moved to another party or time, never removed.
- Variance rule: variance costs more than volume; failures here are silent behind a harmless average.
- Immutable-substrate rule: no overwrite means an indirection layer plus a garbage collector, which costs
u/(1−u)per byte reclaimed. - Sortedness decays: writes consume it; only rewriting restores it.
- Metadata is data: metadata reproduces the layer one level up (the metadata-growth rule forces a hierarchy).
- Deletes are the hardest write.
- Opacity forfeits optimization: wherever a layer cannot see another's meaning, the optimization moves to the side that can.
- Identity versus ordering: stable physical identity and useful physical ordering cannot both be physical; one needs an indirection.
- Slowest reader: reclamation advances at the speed of the oldest reader.
- Unverifiable-assumption rule: ask what a mechanism assumes it cannot verify, and who pays when it is wrong.
- Skip-budget rule (tentative): a layout's total skip power is fixed by the number of units.
Numbers worth remembering
| Quantity | Value |
|---|---|
| Object store minimum efficient request | ~4 MB (L × B), ~40 MB for a 10% latency budget; hence 128 MB row groups |
| Copy-on-write price in Parcelo | U/d = 128,000,000 ÷ 200 = 640,000× |
| Compaction interval | τ* = √(b·U ÷ (a·w)), about 20 minutes in the example |
| Clustering payoff | min(G, 1/s) |
| Z-order floor | U^(1−1/d) units per single-dimension query |
| One synchronous NVMe reader | 4 KB ÷ 80 µs ≈ 50 MB/s, 0.7% of 7 GB/s |
| Zipf cache | top 1% of data gives ~67% of hits; 90% needs ~25% |
| Cache mean latency | 0.9 × 0.1 + 0.1 × 50 = 5.09 ms; 98% from misses |
| Group commit wait | W* = √(2·fsync ÷ λ) |
| Lakehouse planning floor | ~4 round trips × 50 ms = 200 ms per query |
What this mechanism forbids
- You cannot make all queries fast, only choose which.
- You cannot have optimal read, write and space at once (the RUM conjecture is a compass, not a theorem).
- You cannot preserve proximity in more than one dimension.
- You cannot reclaim without establishing that nobody is reading.
- You cannot have both stable identity and useful physical ordering.
The escapes and what they cost. Dual storage (several layouts or sort orders) costs space, write amplification and a freshness gap. PAX costs n decodes per point read. Merge-on-read costs read amplification until compaction. Z-order costs a hard floor on every single-dimension query. Richer summaries cost metadata that recurses. Nested granularity costs complexity. Direct I/O costs you the whole buffer pool.
Failure modes that look like nothing. A new key name in a JSON column; a long-open transaction that stops vacuum; a backfill that destroys the sort order; one sentinel value in a zone map; one hot key in a bucket; one scan that wipes a cache; one reader at queue depth 1. Each has a healthy average. Each is found only by measuring the right number: bytes touched ÷ needed, oldest-reader age, clustering depth, cache miss latency, requests in flight.
Parcelo's final design (Kavya's story): PAX/Parquet files of ~128 MB (3,125 files) on object storage; a key-value copy for the tracking page with a stated freshness gap; delete vectors and merge-on-read with a derived compaction interval; oldest-reader age monitored on PostgreSQL; a global sort by (depot_id, scan_time) with clustering depth watched and reclustering budgeted as rent; reads with 64 or more large range requests in flight and a scan-resistant cache.
Scope. Every mechanism here assumed one node and one writer. Agreement between machines, and the index families as search structures, are outside this mechanism.