Glossary

89 terms, in the sense this mechanism uses them. The platform glossary has every term across all mechanisms.

Absorbing operation: An operation whose repeats change nothing, such as SET x = 5, MAX or delete-by-key.
Ambiguous error: A failure where the caller cannot know whether the operation took effect.
Anti-entropy: A background process that compares replicas and repairs differences; the only repair method that guarantees convergence.
ARIES: A recovery algorithm for databases: analysis, redo of history, then undo with logged compensation.
At-least-once: Delivery where a message may arrive more than once; arises from processing before acking.
At-most-once: Delivery where a message may be lost but never repeated; arises from acking before processing.
Availability: Availability is the fraction of time a service works. 99.9% is "three nines", 99.99% is "four nines". Over a year of 8,760 hours, 99.9% allows about 8.76 hours of downtime and 99.99% allows about 52.6 minutes.1 other meaning →
Backoff (exponential): Waiting longer after each failed attempt, usually doubling.
Backup: A copy decoupled from production in time and authority, protecting against the data being wrong (not only a lost copy).
Blacklisting: Excluding a repeatedly failing node from scheduling, ideally with decay and a quota.
Bloom filter: A bit array and k hash functions answering "definitely not present" or "probably present"A tiny structure that answers "is this key in the set?" with either "definitely not" or "probably yes". It never misses a real member, but sometimes says "probably yes" for a key that is not there (a false positive). It needs about 10 bits per key.
Catch-up condition: Recovery time = backlog / (throughput − arrival rate); recovery needs throughput above arrival.
Category error (exactly-once delivery): Treating exactly-once delivery as a stronger setting of the retry dial; it is impossible, only exactly-once effect exists.
Chaos testing: Injecting failures on purpose at a chosen time to find defects in recovery paths and undeclared dependencies.
Checkpoint: A saved snapshot of all operator state plus source offsets.1 other meaning →A saved snapshot of a job's state plus its position in the input, written to durable storage (storage that survives a crash). After a crash the job loads the last checkpoint and continues from there.
Commit interval: How often a consumer records progress; sets the duplicate blast radius.
Compensation: A new action that offsets an effect that cannot be undone.
Costs are conserved: The trade-off rule that a design which seems to remove a cost has usually moved it: from latency to data loss, from the writer to the reader. The cost is paid in waiting, wasted work or weaker guarantees.A trade-off rule used throughout: a design that seems to remove a cost has usually moved it to another place (from latency to data loss, from the writer to the reader). When we say "the cost is paid in waiting, wasted work or weaker guarantees", we mean these three ways of paying; you will see them again and again.
DAG: Directed acyclic graph: tasks joined by arrows with no loops.Directed acyclic graph: boxes (tasks) joined by arrows (dependencies) with no loops. "Directed" means each arrow has a direction. "Acyclic" means you cannot follow the arrows and come back to where you started. An arrow A → B means "B may start after A finishes."
Dead-letter queue (DLQ): A holding area for records rejected by validation.A place where a pipeline puts records it could not process or did not accept, so they are kept instead of lost.
Dedup window: The time for which deduplication state is kept; duplicates after it are undetectable.
Degraded response: A stale, partial or approximate answer; must be labelled as such.
Deterministic addressing: Naming output by the logical work (not the attempt) so that a retry overwrites the original.
Duplication factor: How many times recovery may repeat an effect.
Epoch: A number that increases at every leadership change and lets others reject stale leaders.2 other meanings →A whole number that increases each time leadership changes. "Epoch 7" means "the seventh leadership period". Every action is stamped with the epoch, so old leaders can be recognized and ignored.
Epoch-based output: Batching output into epochs and making it visible by one designated commit pointer.
Erasure coding (EC): Splitting data into k chunks plus m parity chunks so that any k of n rebuild it.
Exactly-once effect: Observable state changes once even though the message is delivered several times.
Failure detector: A mechanism that decides whether a component has failed, usually from timeouts; it cannot be perfectly correct in an asynchronous system.Any piece of software that decides "that machine is dead". Since a machine cannot announce its own death, every detector works by waiting for a heartbeat or reply that does not come. So every detector is a timeout in disguise.
Failure domain: A set of components that can fail together.A set of components that fail together: machines in one rack share a switch and a power feed; machines in one AZ share a building; every machine running the same buggy deployment shares that bug. When we say two replicas fail "independently", we mean they are in different failure domains.
Failure models: A statement of how components are allowed to fail. Three common ones:
Fan-in: The number of upstream pieces a single output depends on.1 other meaning →A DAG (directed acyclic graph) is a diagram of steps where each step depends on earlier steps and there are no loops. A batch job is often a DAG. Fan-in is the number of earlier pieces one step depends on.
Fencing token: A growing number issued with a lock or lease; the resource rejects requests with an older number.An epoch is a number that only goes up, for example "this is leadership term 7". A fencing token is such a number attached to every request, so that a storage system can refuse a request carrying an older number than one it has already seen.
Full jitter: Sleeping a random time between 0 and the current backoff cap.
Fuzzy checkpoint: A checkpoint taken without stopping writes; consistent at no single instant.
Group commit: Many transactions share one log flush.
Hinted handoff: Storing a write on another node as a hint while the owner is down; yields a sloppy quorum.
Idempotency key: Unique identifier of an intent, used to make retries safe.
Idempotent: Doing it twice has the same effect as doing it once.An operation is idempotent if doing it twice has the same effect as doing it once. "Set the order status to SHIPPED" is idempotent. "Add 1 to the stock counter" is not. Idempotence is the main tool for living with duplicates.
Imposed idempotence: Idempotence obtained by storing a record of what was done.
Independent vs correlated failure: Failures are independent if one does not make the other more likely, and correlated if they share a cause (the same rack, power, firmware). Multiplying failure probabilities is valid only for independent failures.Two failures are independent if one does not make the other more likely. Two disks bought from different vendors in different cities are close to independent. Two disks in the same rack share power, cooling and often the same firmware, so they are correlated. Most of the arithmetic here multiplies probabilities, which is only allowed for independent events.
Jitter: Randomness added to waiting times to avoid synchronized clients.
Last-write-wins (LWW): Merge rule that keeps the version with the latest timestamp; silently discards concurrent writes.Consistent hashing is a way to map keys onto nodes so that adding or removing a node moves only a small part of the keys. A CRDT (conflict-free replicated data type) is a data structure designed so that copies edited independently can always be merged to the same result. Last-write-wins resolves two conflicting versions by keeping the one with the later timestamp and discarding the other.
Lease: A lock that expires after a time limit unless renewed.
Lineage: The map of which assets feed which.A map of what feeds what: table A is read by job J, which writes table B. Following the map shows what would be affected if A changes.
Log (append-only log): A stored ordered sequence of records; consumers track their own offset.An event is a small record: "order 77 placed by user 12 at 12:03:04". A log is an append-only file or service: new events are added at the end, and old events are never changed. A topic is a named log (the word comes from systems like Apache Kafka). Every event has an offset, its position in the log (0, 1, 2, …). A reader remembers "I have read up to offset 5,000", and can go back to offset 3,000 and read again. This ability to replay is the foundation of this whole mechanism. A log keeps events for a configured time (retention), for example 7 days, then deletes the oldest ones.
Low-water mark: The offset below which all records are completed; the safe offset to commit.
Merge rule: The rule that decides which version wins when copies diverge; a precondition of any repair.
Merkle tree: A tree of hashes used to find which ranges of two datasets differ.
MTBF: MTBF (mean time between failures) is the average time a component runs before it fails. MTTR (mean time to repair) is the average time to get it working again.
MTBF / MTTR: Mean time between failures / mean time to repair.
MTTR: MTBF (mean time between failures) is the average time a component runs before it fails. MTTR (mean time to repair) is the average time to get it working again.
Natural idempotence: Idempotence from the operation's own algebra.
Nines: Availability is the fraction of time a service works. 99.9% is "three nines", 99.99% is "four nines". Over a year of 8,760 hours, 99.9% allows about 8.76 hours of downtime and 99.99% allows about 52.6 minutes.
Offset: A consumer's position in a partition (one integer).An event is a small record: "order 77 placed by user 12 at 12:03:04". A log is an append-only file or service: new events are added at the end, and old events are never changed. A topic is a named log (the word comes from systems like Apache Kafka). Every event has an offset, its position in the log (0, 1, 2, …). A reader remembers "I have read up to offset 5,000", and can go back to offset 3,000 and read again. This ability to replay is the foundation of this whole mechanism. A log keeps events for a configured time (retention), for example 7 days, then deletes the oldest ones.
Outbox pattern: Writing the intent to send a message in the same transaction as the data, and delivering it afterwards.
Permanent error: A failure that will never succeed on retry.
PITR: Point-in-time recovery: base backup plus log replay to a chosen moment.
Poison pill: A record that fails deterministically on every retry and can block an ordered partition.
Positional idempotence: Idempotence from writing to an address derived from the logical work.
Prepared transaction: A durable, invisible write waiting for a later commit; it has a timeout.
Primary (leader): The replica that accepts writes and sends them to the followers. Followers apply them a little later (replication lag), so a read from a follower can return older data.A replica is another copy of the same data on another machine. The primary (also leader) accepts writes and sends them to the followers. Followers apply the writes a little later. The delay is replication lag. A read from a follower can therefore return older data than a read from the primary.
Quorum triple (N, R, W): Replica count, read set size and write set size; R + W > N gives overlap.
R: If data is kept on N copies, a write is acknowledged once W copies have it, and a read asks R copies. A quorum is a group big enough that any two such groups share a member. Majority of N is the common case.
Read repair: Fixing stale replicas when a read notices a difference; never fixes cold data.
Rebuild time: Data volume divided by rebuild bandwidth; the window in which further failures cause loss.
Replay: Re-reading records that were already processed.
Replica: A copy of the same data on another machine. The primary takes writes and the followers apply them a little later, so a follower can lag behind.A replica is another copy of the same data on another machine. The primary (also leader) accepts writes and sends them to the followers. Followers apply the writes a little later. The delay is replication lag. A read from a follower can therefore return older data than a read from the primary.
Restore test: Restoring a backup into a scratch environment and comparing row counts and checksums.
Retry budget: A limit that allows retries only up to a fraction of recent successful requests.
RPO: Recovery Point Objective: how much recent work may be lost.Two numbers every recovery plan states.
RPO / RTO: Recovery point objective (data loss allowed) / recovery time objective (downtime allowed).
Rto: Recovery Time Objective: how long until service returns.1 other meaning →Two numbers every recovery plan states.
Sequence number (dedup): A per-producer counter that turns a dedup set into one integer and detects gaps.
Sharp checkpoint: A stop-the-world checkpoint with a fully consistent image.
Sloppy quorum: A write acknowledged by nodes that are not the owners; does not guarantee read overlap.
Split brain: A state where two parts of a system each believe they are in charge.
Standby: A replica is another copy of the same data on another machine. The primary accepts writes; a standby (or follower) holds a copy that can take over.
STONITH: Forcibly powering off or isolating the old node so it cannot act.
The two generals problem: The proved result that over a channel that can lose messages, no finite exchange of acknowledgements gives the sender certainty that its message arrived. Every acknowledgement would need another.Two generals must attack at the same time. They can only send messengers through enemy land, and messengers may be captured. General A sends "attack at dawn". Is A sure that B received it? No. B sends back "received". Is B sure that A got the reply? No. Every acknowledgment needs another acknowledgment. This is a proved result: over a channel that can lose messages, no finite protocol gives the sender certainty that its message arrived. We will use this result as the reason why ambiguity can never be removed.
Time-to-last-successful-restore: The age of the last proven restore; the real backup metric.
Timeout: The maximum time a caller waits before giving up.A timeout is the longest a caller waits for a reply before giving up. A retry is sending the same request again after a failure or timeout. Retries help when the failure was a rare accident. They add load when the failure is caused by overload.
Transactional sink: An output system that supports a durable, invisible write that can be committed later.
Transient error: A failure that may succeed if tried again later.
Two-phase sink commit: The engine prepares writes in the sink and commits them when its checkpoint completes.
Witness replica: A replica that votes and stores metadata only, giving an odd quorum without full data copies.
Write-ahead intent: Recording what you are about to do before doing it, so a scan can finish or undo it after a crash.
Zombie: A worker believed dead that keeps running and writing; fenced by epochs.
δ-CRDT: A state-based CRDT that ships only the recent change instead of the whole state.