Interview questions
Metadata is a set of claims
1What is the difference between load-bearing and advisory metadata?
Load-bearing metadata is needed to read the data at all, such as which files make up a table, so an error shows up on the next read. Advisory metadata only informs people, such as a description or an owner, so its errors are silent and noticed late.
Read moreLoad-bearing versus advisory
2Why does the volume of metadata scale with the number of files rather than the number of bytes?
Each file, partition, column or run is a record to track. A petabyte in 200 files is trivial to describe; a terabyte in 20 million files is not.
Read moreClaims, not bytes
Addressing: which files are the table
3Why should a table's identity not be its name?
Because then a rename is a delete plus a create, and lineage, ownership and usage history silently break. Use a stable, meaningless ID, and resolve names to it late.
Read moreAsset identity
4If you could back up only one, the technical catalog or the business glossary, which would you choose?
The glossary. The technical catalog can be rebuilt by crawling the systems; the glossary records human intent that exists nowhere else.
5What is a partition, really?
A predicate proven true for a set of files without reading them, such as "every row here has date = 2026-10-21". Its value is the fraction of files it lets a query skip.
Read moreA partition is a proof
6What is wrong with encoding partition values in folder names?
The partition value often exists only in the path, so a file placed in the wrong folder gives wrong answers with no error: nothing ever checks the path against the contents.
7Why is the small-file problem a metadata problem first?
Each file written causes a metadata write, whose fixed cost exceeds the small file's own marginal cost. The metastore fails before scans become noticeably slow.
8How does a table format make a change spanning many files atomic on an object store?
It writes new, immutable metadata that nothing references yet, then swaps a single pointer in the catalog with a compare-and-swap. Readers see either the old tree or the new one.
9What limits how often a streaming job can commit to a table?
One successful commit per round trip to the catalog: with a 100 ms round trip, 10 commits a second. So streams commit in micro-batches.
Read moreThe commit point
10When deleting data, do you remove the metadata or the data first, and why?
The metadata first. A crash between the two then leaves orphan files, which are harmless, rather than references to missing files, which make queries fail.
Read moreReconciliation and drift
Describing the contents
11Why are statistics collected by scanning stale exactly where it hurts?
A scan costs time proportional to the data, so it runs rarely, while the tables that change fastest are precisely the ones that change most between scans.
Read moreScan-based collection
12Why do table manifests contain sketches such as HyperLogLog?
Statistics computed at write time, per file, must be combinable across files. Exact distinct counts cannot be combined; sketches can.
Read moreWrite-time collection
13Is branching data as easy as branching code?
Creating a branch, reading it and rolling back are all constant-time, but merging branches with conflicting changes cannot be decided without knowing what the data means: two appends may or may not be duplicates.
Read moreVersioning, branches and tags
14Why can schema-registry clients cache schemas forever?
A schema ID always refers to the same schema, so a cached entry can never become wrong.
Read moreThe schema registry
15Why are some statistics free to maintain while others are not?
Algebraic statistics, such as count, sum, min and max, merge file by file, so new files simply add theirs. Holistic ones, such as the number of distinct values and quantiles, need a full pass, unless you keep mergeable sketches.
Read moreStatistics: whether they mergeNDV from a sample is a hard problem
16What is the danger of stale statistics?
There is no error. Instead: a wrong row-count estimate, a wrong join order, and possibly a 100-times slowdown, with no alert.
Lineage and labels
17What is the right quality metric for lineage?
Coverage: the fraction of all computation that emits lineage at all. Accuracy matters less than knowing what you cannot see.
18Can lineage show that a table is safe to delete?
No. Lineage can prove that a dependency exists, never that none does, because some reads are never recorded. Use a channel nobody can bypass, such as revoking access and logging every denied request.
Read moreFidelity and the absence asymmetryA mandatory channel
19Why is classification propagated along lineage only best-effort?
Labels flow along lineage edges, so they are only as complete as the lineage. Unlabelled data must therefore be treated as sensitive: fail closed.
Read moreThe uncomfortable conclusion
Owners, search, usage and audit
20Why does ownership metadata go wrong faster than any other kind?
People change roles faster than schemas change, and an owner field gives no signal when it stops being true: asserted metadata decays. Derive ownership from load-bearing facts, such as who deploys the code, and resolve it to a team rather than a person.
Read moreOwnership and stewardship
21What is the danger of ranking catalog search results by usage?
It creates a feedback loop that locks in whichever table was popular early, even a deprecated one. Add damping, such as certification or recency.
Read moreUsage ranking is a feedback loop
22Why is 'no reads in 90 days' not proof that a table is unused?
Annual jobs or disaster-recovery uses can sit idle for longer than any threshold, and the longest legitimate gap between uses cannot be observed.
Read moreThe sharp boundary
23How does an audit trail differ from an access log?
Telemetry can be sampled; an audit trail must be complete, tamper-evident and kept for years. Neither can say which rows were accessed, only which queries ran.
Read moreAudit trails versus access logs
24Why is the lifetime of a catalog cache an isolation setting?
It decides how old a version of a table a reader may see.
Read moreClient-side catalog caches