What masking a lost node forbids

What this article forbids

Amortizing redundancy. No batching, compression, caching or approximation reduces the need for a second copy. The only lever is the erasure-coding ratio, paid in reconstruction.
Independence. Every durability figure computed from per-node failure rates is an answer about a world without shared racks, firmware, deploys and operators.
Surviving f failures with fewer than f+1 copies, or reaching majority safety with fewer than f+1 votes (that is, a majority of 2f+1). Witnesses unbundle the two; nothing removes either.
Consistency, availability and low latency together under partial failure. The quorum triple makes the choice explicit and offers no slack.
Protection against logical corruption by adding replicas. It is a different failure class (backup).
Guaranteed convergence from read repair alone. Cold data is never read, so it is never repaired.
Meeting the quorum guarantee with hinted handoff. A sloppy quorum is a deliberate downgrade, not an availability feature.

Escapes and what each costs

Escape	Buys	Costs	Paid by
Add a replica	linear survivability	linear storage and write bandwidth, forever	the storage bill, with no amortization
Erasure coding (durability)	same loss tolerance at ~1.5×	reconstruction cost; degraded reads fan out to k nodes	the tail, whenever anything is down
Witness	the odd vote without the bytes	degrades to zero redundancy, not one spare	the rebuild window
Lower W (async / semi-sync)	write latency off the slowest replica's tail	RPO = replication lag, largest during the failure	the data, at the worst moment
Hinted handoff	write availability during partial failure	the quorum guarantee itself; a backlog aimed at the recovered node	readers, silently
Read repair	repair targeted at wanted data	no convergence for cold data	the data nobody reads
Merkle anti-entropy	convergence at cost proportional to divergence	degenerates to a full scan when divergence is large	the repair path, when most needed

The free escape is placement: a copy in a different failure domain costs the same as a copy in the same rack and is worth several times more. It is a design decision with no runtime cost, and it is the last thing teams do. This is the third part in a row in which the zero-cost escape is the one reached for last.

Case, resolvedConstructed case · Wanjiru and Dmitri at Mediflow

Two of three copies in one rack

A rack power loss took two of the patient database's three copies, and management asked whether a small third site could replace a full copy. Where it started: Durability and availability →

The rack. Dmitri redraws the placement. The three replicas now sit in three different failure domains: one in each Nairobi site on separate power feeds, with the cheaper Mombasa site hosting a witness (votes, no bytes). To management's question he answers with arithmetic. Two copies plus a witness cost about 2× storage instead of 3×, a saving of a third. But losing one data site leaves one copy. Rebuilding a 4 TB database over a 200 MB/s link takes 4,000,000 MB ÷ 200 MB/s = 20,000 s, about 5.6 hours, and in that window a second failure is total loss. Management chooses three full copies for the patient records and the witness design for the less critical reporting database. The "eleven nines" slide is filed with a note: it assumes independence, and Mediflow's rack proved otherwise.

Convergence. Hinted handoff is switched off for the patient records: Mediflow prefers a failed write to a silently weakened quorum. A background Merkle-tree repair runs weekly.

Across all four parts, Mediflow now has a system that keeps answering while a node is gone (this article), a way to redo interrupted work within a known time (Recovery mechanisms), and a way to make the redo safe (Making a repeat harmless). What it redoes from, the durable log and the backups, is Persistence & Durability's work.