The fidelity ladder

Which detection methods exist, and what does each rung down give up?

CaseConstructed case · Tariq at Quillpost

Cancelled orders still on the dashboard, and a full disk

For orders, Tariq takes the cheapest route. He writes a query that runs every 15 minutes and copies rows where updated_at is later than the last time he looked. For a week it looks fine. Then three complaints arrive:

Finance says the dashboard still counts 212 orders that customers cancelled and support deleted from the database.
Support says the dashboard shows orders as "shipped" but has no record that they were ever "packed". The audit report needs every status step.
Lena ran a correction script that fixed 1,800 wrong prices. The warehouse still shows the old prices, and nothing will ever fix them.

Tariq switches to reading the database log. It works for a while. Then, one Monday, Lena calls: the production database disk is 99% full, and the cause is Tariq's CDC reader, which crashed on Friday evening.

How it ends →

There are several ways to detect a change. Imagine them as a ladder. Each rung down is cheaper and less coupled to the source, and loses a nameable class of change. Nothing recovers a class of change that the source never recorded.

Method	Cost to source	What it can miss
Log-based CDC	log volume, retention coupling	writes that bypass the log (rare)
Trigger-based CDC	write latency on every statement	writes that bypass triggers; bulk loads
Query/diff-based CDC (polling)	almost nothing	hard deletes, intermediate states, writes without a timestamp
Watermark-partition (append-only data)	nothing	late data in a partition already processed, restatements of old partitions
Snapshot diff (table format)	nothing: metadata only	which rows changed inside a rewritten file; history the cleaner has expired
Outbox (the writer announces it)	one more row per event, written in your transaction	writes that do not go through the code that fills the outbox