Evolution operations and expand/contract
Which operations are safe, and why does a rename take six steps?
The table of safe operations is not a convention. It follows from one criterion:
("Information-preserving map" means: every old value is converted to a new value, and no two different old values become the same. This is total and injective from "Before you start" on the overview.)
| Operation | Safe? | Why |
|---|---|---|
| Add optional field with default | Both directions | An absent value maps to the default in a fixed way; old readers ignore the new field |
| Add required field | Never | Old data has no value for it, and "no default" is by definition "no mapping" |
| Drop optional field | Backward only | Old readers may still reference the field |
| Widen a type | Yes | Injective and total over the old domain |
| Narrow a type | Never as a schema change | Not total: some old values do not fit. It is a data migration wearing a schema edit's costume |
| Reorder | Free under name or ID identity; fatal under position | See Identity by name and by ID |
| Rename | Entirely decided by the identity rule | ID: metadata only. Name: drop plus add. Position: no effect |
(In some formats a dropped field that has a default is safe in both directions, as in the A to B step above. The table row describes the general case: if an old reader has no default for the field, it breaks.)
Why rename needs several steps
Under name identity, rename is "drop plus add". Drop breaks readers. So the only safe rename is the expand/contract protocol (also called parallel change):
- Expand: add the new field (
weight_kg) next to the old one (weight). - Dual-write: write both fields in every new record.
- Backfill: fill the new field for old rows.
- Migrate readers: move every reader to the new field.
- Verify: prove that nobody reads the old field.
- Contract: drop the old field.
A rename is therefore not one operation. Treating it as one operation is the exact mistake Ravi made. The procedure exists to correct that mistake.
The hard step is the last one
Dropping requires proving nobody reads the column. Proof requires read telemetry: query-log mining or column-level lineage. Organizations without it never complete the contract phase. That is why mature tables collect dozens of dead columns. It is not laziness. It is the absence of the observation mechanism that would make the last step decidable. It is the cleanest example here of a technical decision blocked by a missing source of information, not a missing feature.
expand/contract for weight -> weight_kg (timeline)
week 0 schema: weight writers: weight
week 1 schema: weight, weight_kg writers: weight AND weight_kg (expand + dual-write)
week 2 backfill old rows: weight_kg = weight
week 3-6 readers move one by one to weight_kg
week 7 query log shows no read of `weight` for 30 days (verify)
week 8 schema: weight_kg (contract / drop)Safe change means an injective map over existing data. Rename is expand/contract, and you can only finish the contract if you can observe reads.