Summary and key takeaways
The six articles in one sentence each:
- A cache is stale by construction. Hit rate h gives leverage 1/(1 − h) (20 at 95%), so a working cache becomes a hidden dependency; the benefit depends on the workload (C ≈ N^h at alpha = 1), not on the code.
- Cacheability comes from reuse distance. Measure miss-ratio curves (sample by key hash), find knees and plateaus, decompose the stream into stationary, bursty and one-shot parts, and check that a hit is cheaper than the tier below. No cache improves any percentile above h.
- What you cache is a ladder of derivations. The farther a result is from the source, the lower its reuse. Metadata is the best deal; materialized views and algebra (monoids, sketches) make results maintainable; negative caching with a Bloom filter protects against missing keys.
- Freshness is bought with coordination, staleness or verification, and nothing else. Write routing and freshness bounding are independent. Delete, do not update; keep a TTL floor; version keys; protect against stampede; jitter TTLs; ramp warm-ups from origin capacity.
- Eviction cannot beat a proven bound, so improve the request stream: admission, hints, bypass. Use cost-aware policies and partitions when miss costs differ; use consistent hashing with vnodes and a client near-cache for hot keys.
- The design forbids nine things; each escape moves its cost to someone. The costs of caching are the hardest to attribute of any mechanism across the platform.
Key formulas:
| Formula | Meaning |
|---|---|
| Leverage = 1/(1 − h) | origin load multiplier when the cache is lost (20 at h = 0.95) |
| C ≈ N^h (alpha = 1) | cache size needed for hit rate h over N items |
| h > t_new / t_below | a new tier must beat the tier below it |
| L = λ_k · W | duplicate recomputes in a stampede |
| (1 − h(t)) · λ(t) < origin capacity | warm-up and reshard ramp |
| Hit-rate loss from versions ≈ ln V / ln N points | cost of V live versions per key |
| Modulo hashing moves (N−1)/N of keys; ring moves K/N | why membership changes need consistent hashing |
| Imbalance ≈ 1/√V | virtual nodes |
The nine prohibitions (what no design can do): free freshness over a bypassable copy; hit rate bought linearly; optimal online eviction; exact reuse decisions over cached query fragments; exact invalidation without the derivation graph; cross-entry consistency without one designated version cell; improving a percentile above h; splitting one hot key across nodes by hashing; caching a workload with no reuse.
The escapes and who pays: immutability and content addressing (the writer and the garbage collector); materialized views (the writer); algebra and sketches (the reader's precision); verification (the reader, cheaply); randomization (diffuse); admission sketches (one-shot traffic); partitioning and pinning (idle capacity); tiering (the misses); client near-cache (staleness).
Patterns that repeat:
- Immutability deletes problems: no invalidation and no write-routing choice.
- Randomness replaces coordination: early expiry, jitter, vnodes, sampling.
- A hit that writes is a cost: LRU bookkeeping contends on the read path.
- Correlation is the enemy: identical TTLs, shared hot keys, one tenant's scan.
- Cold start and reshard are the same event, and one ramp controller serves both.
Two questions for every cache you meet: What does it assume that it cannot verify (no unknown writers, complete dependency sets, stable alpha, independent TTLs, a store that can absorb (1 − h)λ)? And who pays when that assumption fails?
Ifeoma's runbook line (from Mateus): "Every cache is a bet that nobody changed the data and that the database can survive without it. Write down both bets."