Designate one write
Atomicity over n items is atomicity over one designated item.
What it is
The same rule that guides this whole subject also applies here: atomicity over n items is implemented as atomicity over one designated item. A table format does this with one snapshot pointer. For freshness it becomes a testable claim. Freshness over n cached entries needs one designated "version cell" if the entries must agree with each other. A per-entry TTL uses no such cell. Look at six designs:
From Caching › One designated cell
All five end at the same requirement: the effect and the record of the effect must become durable together. If they do not, the ambiguity only moves down one level. This is the principle "designate one write": atomicity over n things is always built from atomicity over one thing. If you need many things to happen at once, you pick one single write (a commit record, a rename, a pointer swap) whose success makes all of them count. This article applies that principle five times in a row. The mechanisms look different, and they are not.
From Fault Tolerance & Delivery Semantics › Where 'already done' lives
The single most useful saga rule is "atomicity over n items is atomicity over one item" in disguise. Put the irreversible step at the end. If the step that cannot be undone happens last, it becomes the designated event that decides whether everything before it counts. You have recovered a commit point by construction instead of by protocol. Where that is impossible, defer the irreversible effect (hold the email; authorize the card payment but do not settle it) until a point that can serve as the commit.
From Transactions & Concurrency Control › Sagas
Where it appears
In 13 mechanisms. Each says what the mechanism is about, then how this principle shows up there, in its own words.
- Persistence & Durability
When a system says "yes, your data is safe", what exactly has it promised, and what did it have to do to keep that promise?
Principle: when the storage's atomic unit is at least as large as your object, torn-write protection disappears and the whole problem becomes one pointer swap.
Read it inTorn pages and double-write buffers
- Transactions & Concurrency Control
What happens when many users change the same data at the same time, and how do we keep the result correct?
Percolator removes the coordinator by putting the decision where readers already have to look.
Read it inRemoving the coordinator: Percolator and CalvinSagasCompare-and-swap: the object store's only transaction primitiveNothing below you can do many writes at once
- Caching
What happens when we deliberately keep a second copy of data that no protocol forces to be correct?
A single designated version cell is necessary for cross-entry consistency over cached derivations.
Read it inOne designated cell for many entriesWrite routing, and why you delete rather than update
- Distributed Systems Coordination
Who is allowed to decide, and how does everyone else learn the decision without asking?
The map carries a version, and the version is the fencing token.
Read it inResharding: the map is harder than the bytesAgreement is expensive, dissemination is cheap
- Fault Tolerance & Delivery Semantics
What is the cheapest thing that can be safely done twice?
But there is an escape that is not an ordering, and it is the correct answer: store the offset in the sink.
Read it inConsumer offsets and commit orderingAt-most-once, at-least-once and the exactly-once category errorWhere the memory of 'already done' livesIdempotency: the three sourcesRollback and undo: where reversing stops
- Scheduling & Workload Management
Which work runs now, which work waits, and who pays for the waiting?
The scheduler is itself a shared mutable data structure.
Read it inThe scheduler is shared state
- Change Data Capture & Incremental Processing
What does it cost to know what changed, and is that cheaper than not needing to know?
All resolutions reduce to one rule: make one system the source of truth and derive the other.
Read it inReplay and dual-write avoidance
- Table & Storage Maintenance
Who pays for the gap between the order data arrives in and the order it is read in: the writer, every reader, or a background job, and what is the break-even?
So it commutes (can be reordered) with any commit that only adds files.
- Schema Management & Evolution
What happens when the shape of your data changes on its own schedule, and nobody can change the readers, the writers and the old data at the same moment?
ID identity works only under one rule: an ID is never reused.
Read it inIdentity by name and by ID
- Data Quality & Validation
How do you check that data is right, when "right" is a claim about the world and not about the bytes?
The validation pattern followed the storage primitive, not the other way around.
Read it inA gate is a check with authorityTraining-serving skew: one number computed twiceWhat reconciliation forbids
- Metadata, Catalog & Lineage
What can a system prove about its data without reading the data, and how does it know that proof is still true?
The catalog is the linearization point of the table.
Read it inThe commit point
- Orchestration & Workflow
How do you make unreliable, unobservable, irreversible side effects behave like a pure function of a time interval?
The general form: the interval key is an addressing scheme.
Read it inCron, calendars and data intervalsRetries, the unknown state and fencing
- Interfaces & Formats
What can two parties agree on when they cannot talk to each other?
A source is only as resumable as its position is total and monotonic.
Read it inConnector frameworks, ELT and ETLFile drop, backfill, and what integration forbids
Related
Made on the same pages most often.