Playbooks

Choose a lookup structure

You need to find records by key: a static structure, a hash table, a skip list, a B-tree or an LSM-tree? Four questions, each removing more of the design space, then a check of the failure mode.

Choose an ordering structure

A query needs data in order: a heap, an existing order, an in-memory sort, an external sort or a distributed sort? Four questions, starting with how much order you really need.

Choose a sketch

An exact answer is too big to keep: which sketch answers your question (membership, distinct count, frequency, quantile, similarity or a sample), and will it survive merging, duplicates and the questions asked later?

Choose a bit-level structure

A structure is read-mostly and could be packed far smaller: is the saving worth it, and should it be a bitmap, an Elias-Fano list or a packed array with rank/select? Plus when min/max statistics can prune at all.