Labs
Each lab takes under an hour on a laptop and turns a claim from this mechanism into a number you measured yourself.
- 1
Checkpoint interval optimum
A 30-day job with Poisson failures: sweep the checkpoint interval, find the optimum against √(2·C·MTBF), then get the MTBF wrong by 4×.
SimulatePython - 2
Retry storm: no jitter versus full jitter
400 clients retry against a server that was down for 2 s: exponential backoff with no jitter, equal jitter and full jitter.
SimulatePython - 3
The catch-up condition
A 60-second outage at 500 arrivals per second, recovered at throughputs from 525 to 2,000: recovery time against utilisation.
SimulatePython - 4
Replay determinism check
An aggregation run twice over the same events, with now() and a mutable lookup, then with event time and a versioned lookup: compare output hashes.
MeasurePython - 5
Duplicates versus an idempotency key
A small payment service, 1,000 charges and 20% lost replies: a new key per attempt, an intent key with a stored outcome, and suppression with an error.
Real systemPython - 6
Dedup window size and the Bloom filter trap
The memory of exact deduplication for an hour and a week at 100,000 keys/s, then a Bloom filter alone and in front of an exact set.
MeasurePython - 7
Idempotent sink with a conditional insert
Redelivered batches into SQLite: a plain insert duplicates, a keyed conditional insert does not, and check-then-insert races.
Break itPython - 8
Commit ordering and the offset in the sink
A consumer adds 1..10,000 to a total and crashes 20 times: position saved before processing, after, and in the same transaction.
Break itPython - 9
Auto-commit versus commit-after-process
Kafka auto-commit against commit-after-process, each killed once mid-run: silent loss against a bounded number of duplicates.
Real system · Break itDocker, Python with kafka-python