Labs
Each lab takes under an hour on a laptop and turns a claim from this mechanism into a number you measured yourself.
- 1
Planning time versus partition count
Read the metadata of 1,000 to 1,000,000 partitions from SQLite, with and without a filter, and size the planning slots with Little's Law.
MeasurePython - 2
A file in the wrong folder gives wrong answers, silently
Copy one day's Parquet file into the next day's Hive folder and read the table; then store the date inside the file and count mismatches.
Break itPython with DuckDB - 3
The pointer swap: two writers, one winner
Two writers commit table versions through a pointer: with an exclusive create (compare-and-swap), then with a plain write.
Break itPython - 4
Read the statistics inside a Parquet file
Read Parquet footer statistics with parquet_metadata: files skippable for a one-hour range, sorted against random; merge two files' statistics by hand.
MeasurePython with DuckDB - 5
Lineage coverage: static parse against runtime log
Lineage for 20 small jobs by parsing their SQL and by recording what DuckDB actually scanned: edges found by each.
MeasurePython with DuckDB, sqlglot - 6
The ranking feedback loop
52 weeks of catalog search where most users take the top result: does the better table ever win, and what certification bonus does it take?
SimulatePython