Labs
Each lab takes under an hour on a laptop and turns a claim from this mechanism into a number you measured yourself.
- 1
Per-value checks ride on the parse
Time parsing a 5-million-row CSV alone, with checks in the same pass, and with a separate checking pass.
MeasurePython - 2
The false-alarm floor of a cross-table check
Two loaders on different schedules: the orphan-order rate a referential check reports, and how allowed lateness removes it.
SimulatePython - 3
P-values measure your row count
A 0.07% shift tested with KS at 10³ to 10⁷ rows: the p-value collapses with n, the PSI does not.
SimulatePython with NumPy, SciPy - 4
Alert fatigue by arithmetic
12,000 checks at a 0.001 false-alarm rate for 90 days, with and without a Bonferroni correction, against weekly real defects.
SimulatePython with NumPy, SciPy - 5
How many rows must you sample, and what does blocking cost?
How large a sample finds a rare defect, against 1 − (1 − p)ⁿ and n ≈ 3/p, and how blocking caps the recall of duplicate search.
SimulatePython with NumPy - 6
Find five bad rows among a million with a Merkle tree
Find 5 changed rows among a million with a Merkle tree, then see why a XOR table hash cannot see duplicated rows.
SimulatePython