Data at rest and the one ratio
What is different about data at rest, and which one ratio explains most of the cost?
Data at rest: the consumer is unknown
Think about data in motion, such as a message traveling from one service to another. Someone is waiting for it right now. You know the consumer, and the order in which messages arrive may matter.
Data at rest is different. The moment data stops moving and is stored, the consumer becomes unknown. It is not only unnamed. It is unbounded in time. You write once, and the data is read by queries that have not been written yet, over years, with filters that nobody has thought of.
That difference is this whole mechanism:
Two things change places when data comes to rest.
What stops mattering: sequence. In a stream of messages, arrival order can carry meaning. At rest, a relation (a table) is a set. Any reordering of its rows is the same table. Physical order carries no meaning, so it is yours to spend. Layout is a design space only because the logical model does not constrain it.
What starts mattering: the I/O quantum becomes a design surface. The hardware fixes the size of the unit it reads and writes (a page, a block, an erase unit). But what goes inside that unit is your decision. Remove the vocabulary and every mechanism here does one thing: it decides what is adjacent to what.
The governing question of this mechanism:
And the central idea in one sentence:
The one ratio: bytes touched ÷ bytes needed
Almost everything here reduces to one ratio:
The denominator (bytes needed) is set by the query. The numerator (bytes touched) is set by the layout and by the storage medium. That is why one layout decision can be worth a few times on one medium and a hundred thousand times on another.
Now put the same row in an object store. A request has 20-100 ms of fixed latency and a price per request. The cheapest sensible read there is somewhere between 1 and 10 MB. Reading 1 MB to get 200 bytes: 1,000,000 ÷ 200 = 5,000×. Reading 10 MB for 200 bytes: 10,000,000 ÷ 200 = 50,000×. So the range is about 10⁴ to 10⁵×.
Nothing about the row changed. The fixed cost per operation moved up a level, and it dragged the natural unit of access up with it. This is the batch-size rule: when each operation has a fixed cost, the natural unit of work is the fixed cost divided by the cost per byte. Below that size you mostly pay the fixed cost. It is the reason a Parquet row group is about 128 MB and not page-sized. That number is not a taste decision. It is a fixed cost divided out.
Keep this in mind:
- On local NVMe (a fast SSD), layout decisions buy you 10-100×.
- On object storage, the same decisions buy you 10³-10⁵×.
The layer gets more important as the storage gets cheaper and more distant, not less.
The shape of the layer: three decisions and one price list
This mechanism has four groups of mechanisms. Three of them are independent decision variables. The fourth is not a peer: it is the cost function that the other three are judged against.
| Decision | Question it answers | Main amplification it governs | Part |
|---|---|---|---|
| Adjacency | Which values are stored beside which? | Read amplification (and the compression ratio, which follows from adjacency) | Layout models |
| Mutation | What happens physically when a value changes? | Write amplification and space amplification | Mutation strategies |
| Placement | Which unit does a record go into? | Which units can be skipped entirely | Physical clustering |
| Substrate | What does one touch cost, and at what granularity? | None. It sets the exchange rate for all three. | Access and media |
The mismatch rule ties the three decisions together. There are three granularities in play: the write unit, the read unit and the update unit. Each mismatch between two of them creates one named amplification: write unit versus update unit gives write amplification, read unit versus bytes needed gives read amplification, and keeping old versions gives space amplification. The three amplifications are not three phenomena. They are one phenomenon counted three ways.
Adjacency versus placement. Adjacency attacks read cost from the inside: when you touch a unit, waste less of it. Placement attacks read cost from the outside: never touch the unit at all. They complement each other. Placement is strictly more powerful when it works: skipping a 128 MB row group beats decoding 3 of its 40 columns. But it works only when the query's filter matches the physical order, and that match is exactly the bet you cannot hedge.
Mutation is a different axis. It asks whether the stored layout is a function of the current state or of the history. Update in place means current state. Log-structured storage, multi-version storage (MVCC) and merge-on-read mean "history plus a rule to rebuild the current value". This one choice decides whether a read is a simple lookup or a merge, and whether the background cleaner is a vacuum (frees dead rows) or a compactor (merges and rewrites files).
The RUM conjecture, and how far to trust it
You will often meet the RUM conjecture: you can optimize at most two of Read cost, Update cost and Memory (space) overhead, never all three. Use it as a compass, but know what kind of claim it is. It is a conjecture, stated as such by its authors (Athanassoulis et al., 2016), and it has no proof. In some situations all three improve at once. For example, better compression helps read cost and memory together, and dictionary encoding can help updates too. RUM is a strong pattern at the frontier of what is possible, not a law of nature.
The statement that really cannot be avoided is weaker and sharper:
That is the conserved-cost rule in a storage costume: a design that seems to remove a cost has moved it, to another time, another party (writer, reader, background job, the application) or another resource.