What delivery semantics forbids

What this article forbids

Exactly-once delivery. It is the Two Generals problem. Not an engineering gap: a proof.
Atomicity across two systems without a shared commit log or a blocking protocol.
Telling "not done" from "done, ack lost". Permanently.
Exactly-once for a non-idempotent external effect. No architecture, no framework, no vendor can do it.
Recovering a record you did not retain. Retention is the hard boundary of every recovery story.

The escapes and their prices

Escape	Buys	Costs
Make the effect repeatable (Fault tolerance)	effectively-once, whatever the transport	a key and a store, or a redesigned sink
Replay	fixes past wrongness, which nothing else does	retention cost, determinism discipline, pacing, blast radius
Idempotency by replacement	a replay that cannot count twice	a sink that can overwrite a partition or swap atomically
Version-based application	replayed and late records cannot overwrite newer state	a version or LSN in every record
A record envelope (`event_id`, `version`)	the one guarantee that crosses every system boundary	one field each, from the first day
Reconciliation counts per hop	loss becomes visible	a job and an alert per hop

The one sentence

Case, resolvedConstructed case · Lena at Cartwheel

A replay doubled last month's revenue

A three-week replay to fix a currency bug doubled most of the replayed days and nudged untouched ones, and the warehouse held 0.6% fewer orders than Orders produced with no error logged. Where it started: Delivery and processing semantics →

The doubled revenue. The revenue job deduplicated on event_id with a 24-hour window. A replay of three weeks arrived long after the window had forgotten the keys, so every replayed order older than a day was counted again. Lena changes the job to rebuild by replacement: it recomputes each day's partition from the log and swaps it in atomically, so a replay overwrites instead of adding. The few cents on untouched days came from the transform itself, which looked up today's currency rate. It now uses the rate valid at the order's time, from a versioned rate table, and a replay of one known week into a scratch table must match before any real replay runs.

The missing 0.6%. Lena counts records in and out at every hop. The gap is at the enrichment step: a call to the seller service sat in a try/catch that logged a warning and committed the offset anyway. Every timeout was an at-most-once decision that nobody had made. A failed enrichment now goes to a retry topic, and a daily reconciliation count per hop raises an alert when in and out disagree. Kafka's exactly-once had been working all along; the loss was outside its boundary.

Omar asks what the lesson is. Ravi writes it under his earlier note on the wall: "Exactly-once is a property of a boundary. Ask where the boundary is, and make every effect outside it repeatable." The record envelope for every Cartwheel event now carries an event_id and a version from the start.

Lena's last remark, at the end of the project, is the sentence that closes this mechanism: moving data is the act of deciding what you are willing to lose. After six problems, Cartwheel can finally say, for each flow, what it is willing to lose, and that someone chose it.