Read-optimized vs. write-optimized
Where you meet it: B-tree vs. LSM, merge-on-read vs. copy-on-write.
What it is
Writing is easy if you do not care where data lands: add it at the end. Reading needs to know where things are, which needs structure (sorting, an index). And structure must be kept up to date when data changes. So the work of "knowing where things are" is paid at write time, at read time, or in space.
This three-way trade is the RUM conjecture (Read, Update, Memory — you can optimize at most two). It explains why read-optimized and write-optimized designs are opposites, and why every physical layout is a bet on an access pattern: data sorted by user_id is not sorted by timestamp.
From Physical Substrate & Performance Laws › Seven root causes
You will often meet the RUM conjecture: you can optimize at most two of Read cost, Update cost and Memory (space) overhead, never all three. Use it as a compass, but know what kind of claim it is. It is a conjecture, stated as such by its authors (Athanassoulis et al., 2016), and it has no proof. In some situations all three improve at once. For example, better compression helps read cost and memory together, and dictionary encoding can help updates too. RUM is a strong pattern at the frontier of what is possible, not a law of nature.
From Storage: Physical Organization › Data at rest and the one ratio
Where it appears
In 11 mechanisms. Each says what the mechanism is about, then how this trade-off shows up there, in its own words.
- Physical Substrate & Performance Laws
Why do trade-offs exist at all?
This three-way trade is the RUM conjecture (Read, Update, Memory — you can optimize at most two).
Read it inSeven root causes: why trade-offs exist at allDisk and flash
- Data Movement
How do records cross between two systems that have independent rates, independent availability and independent lifetimes, and what are you willing to lose when the assumption underneath the crossing breaks?
You deliberately traded read simplicity for write distribution.
Read it inPartitioning: the four strategies
- Storage: Physical Organization
What should sit next to what, given that I must decide at write time and find out at read time?
You will often meet the RUM conjecture: you can optimize at most two of Read cost, Update cost and Memory (space) overhead, never all three.
Read it inData at rest and the one ratioLayout models: row, column and PAXDeletes, versions and reclaiming spaceMutation strategies: what happens when a value changesPartitions, buckets, row groups and zone maps
- Core Data Structures
How do we choose a data structure when the real cost is set by the hardware, not by Big-O?
The classic statement is the RUM conjecture (Read, Update, Memory; Athanassoulis et al., 2016): optimize any two and the third gets worse.
Read it inThe big picture: a data structure is a betPutting the pieces together: the five currencies and the four compositionsEpoch-based reclamation and hazard pointersLSM-treesBalanced trees (red-black, AVL)
- Encoding, Serialization & Compression
What do we turn our values into when we turn them into bytes, and what does that choice cost?
For ZSTD-family codecs, decode speed is roughly independent of the compression level.
Read it inBytes become an outputGeneral-purpose compression: the last passBit packing, front coding, nulls and floats
- Indexing
When does a separate "where is it?" structure pay for itself, and what does it cost to keep it true?
Bitmap indexes belong to read-mostly systems.
Read it inBitmap indexes: making selectivity multiplicative in the indexThe consistency boundary, local and global indexesWhy indexes exist and when they winDesign an index
- Caching
What happens when we deliberately keep a second copy of data that no protocol forces to be correct?
Write-through is also an admission decision, and it admits on write locality rather than read locality.
Read it inWrite routing, and why you delete rather than updateThe governing arithmetic: two inequalities
- Table & Storage Maintenance
Who pays for the gap between the order data arrives in and the order it is read in: the writer, every reader, or a background job, and what is the break-even?
Larger steps mean less amplification, but they also mean a longer wait before data is compacted, and bigger jobs that clash more often with writers (Running maintenance safely: concurrency, triggers, deletes, verification).
Read it inCompaction as bin-packing
- Data Modeling
What does one row mean, who is allowed to change that answer, and what breaks downstream when they do?
Read the table as a single statement: every read-optimized tree encoding pays proportionally to subtree size on a move.
Read it inHierarchies
- Specialized Processing Models
What do you build when a query predicate cannot be reduced to a range over an ordered key?
Compression becomes a write-path operation, and the chunk becomes immutable as a side effect.
Read it inCompression as the write path
- Interfaces & Formats
What can two parties agree on when they cannot talk to each other?
In one sentence: columnar files are readable only when complete, because the price of random access into a file is a footer, and a footer means an ending.
Read it inColumnar files and in-memory formatsTable formats and log formats
Related
Made on the same pages most often.