What encoding forbids
What can no encoding give you at the same time, and what does it hand to whatever reads its bytes?
What this mechanism forbids as a whole
Across the three stages (serialization, lightweight encoding, general-purpose compression), you cannot have all four of these at once:
- minimum bytes,
- self-sufficient decoding (the bytes carry everything the reader needs, with no registry or dictionary outside),
- random access at the granularity of one value,
- in-place update.
Each stage gave up a different one.
- Serialization traded self-sufficiency against density, and made the trade visible as a registry.
- Lightweight encoding kept random access, and gave up updatability completely.
- General-purpose compression gave up random access, to reach the entropy floor.
Nothing here gets all four. Every format you will ever evaluate is a named position in this four-way impossibility.
What this mechanism hands upward
The output of this mechanism is a byte stream. Whoever builds on top inherits its properties and cannot change them:
- a fixed decode granularity (the smallest piece you can decode),
- a fixed skipping granularity (the smallest piece you can skip),
- a set of columns that either do or do not support predicate pushdown,
- a corruption blast radius.
Storage layout can only use what the encoding preserved. Zone maps work only because someone sorted. Pushdown works only because someone chose a dictionary. Parallel scan works only because someone accepted the loss of ratio at frame boundaries.
A last point connects the two mechanisms. The choice of physical clustering (which rows sit next to each other on disk) is made in storage layout, but its whole payoff is measured here, in ratio and in skipping. The two are one decision seen from two sides: the write-side choice and its read-side return.