What correctness under failure forbids

What this article forbids

Exactly-once delivery. Permanent, from the two generals result. No protocol closes it.
Preventing a duplicate at the sender. The ambiguity is the sender's; the state is the receiver's.
Exact dedup over unbounded delay with bounded state. Every window is a bet that is correlated against you.
Exactly-once over an effect the system does not own. Side effects in user code are outside every boundary.
A correct commit ordering when position and effect live in separate stores. Only 2PC or idempotence helps, never a better order.
Approximate dedup without risking loss. The false positive points the wrong way.
Telling a poison pill from a transient fault in general. Attempt count is a proxy that misfires during incidents.
A writer-enforced visibility guarantee. Uncommitted output is physically readable; the rule lives in the reader.

Escapes and what each costs

Escape	Buys	Costs	Paid by
Absorbing algebra (SET, MAX, delete)	idempotence for free	only absolute operations; relative ones need CAS	the concurrency path
Deterministic addressing / overwrite	idempotence with no dedup state	the unit of work must have a stable name	pipeline design, once
Idempotency key + stored outcome	works for arbitrary effects	unbounded state, so a window that is a silent bet	the data, silently, when the window is short
Sequence numbers	O(producers) state, and gap detection	requires ordering and durable producer identity	autoscaling, which breaks identity
Offset in the sink	deletes the second write entirely	couples position to the sink schema; one sink only	portability
Two-phase sink commit	exactly-once with a foreign sink	visibility latency = commit interval; timeout fragility	the consumer's latency budget
Epoch-based output	O(1) state, plus zombie fencing	all-or-nothing re-execution; readers must comply	anyone reading the raw path
DLQ / skip	the pipeline keeps moving	completeness broken with no signal	downstream aggregates, silently
Stop-the-line	correctness preserved	availability, and a human on the critical path	the service-level target

The free rows have the same shape as in earlier parts: choose the algebra, and name the work rather than the attempt. Both are design-time decisions that cost nothing at runtime, and both are reached for only after the expensive mechanisms have failed.

Case, resolvedConstructed case · Wanjiru at Mediflow

Two texts, two charges, one poison message

Patients still got two 'results ready' texts, a few were still charged twice, and one malformed lab message blocked a partition for eight hours while the consumer crashed and restarted on it. Where it started: Making a repeat harmless →

Wanjiru walks through the decision procedure (Make a repeat harmless) with Dmitri, one effect at a time.

The database row. The consumer now stores its offset in the same PostgreSQL transaction as the patient_records row (question 2). A crash between the two writes is no longer possible. A replayed message finds next_offset already advanced and does nothing.

The card charge. The payment provider accepts an idempotency key. She stops generating a key per attempt. The key is now the lab result id, sent on the first attempt and on every retry (question 4, and the rules of Source 3). The provider stores the key with the outcome, so a retry after a timeout returns the original "charged" answer. The double charges stop.

The SMS. The SMS provider has no idempotency key and no prepared writes. This effect is outside what Mediflow owns. Wanjiru says so honestly in the design document: the SMS may be sent twice after a crash in the narrow window between the provider call and the commit. She reduces the window by recording "SMS sent" with the result id before the call returns to the main loop, and she accepts the rare duplicate text message. Nobody pretends the text message is exactly-once.

The poison pill. She adds a retry counter to the message headers, not to memory. A parse error is classified as permanent at the point of failure. Priya and Dmitri make the business decision: for the billing ledger, a bad record stops that partition, and an alert wakes the owner. For "results ready" notifications, the text is skipped, the lab result still goes to a quarantine topic, and the DLQ depth is graphed with a named owner. A dependency outage no longer counts toward the poison-pill threshold, because connection errors are classified as transient.

Dmitri adds one more check: the payment provider's idempotency window is 24 hours, and the longest recovery they have ever seen is 3 hours. The bet on maximum duplicate delay is written in the runbook.