Scenarios

At a 95% hit rate the database saw 5% of 30,000 = 1,500 reads per second, and it was sized for that. An empty cache sends it all 30,000: 20 times its normal load, the cache's leverage of 1 / (1 − 0.95).

The database did not fail on its own. Its capacity plan silently included the cache, and the restart removed it.

Warm the cache before it takes traffic; restart nodes one at a time; cap the database's concurrency and shed load so it survives a cold start; consider a persistent or second-tier cache that survives restarts.

Warming costs time and memory. Whatever you choose, write the cache into the capacity plan as a dependency.

Build the miss-ratio curve, the hit rate at every cache size, from production traffic, using reuse-distance sampling such as SHARDS. If the curve is flat around the current size, more memory buys nothing.

p99 lies above the 95% line, so it is made of misses. To move p99 the hit rate would have to exceed 99%. With skewed (Zipf, α = 1) popularity over N keys, the cache needed for hit rate h grows like C = N^h, so each extra point of hit rate costs a multiplicatively larger cache.

Work on the miss path instead: add a tier below, or hedge slow reads. Grow the cache only to a knee in the curve, and only if the curve shows one.

A remote hit costs 0.3–1 ms; a local NVMe read about 0.1 ms. The proposed tier is slower than the one above it.

A new tier pays only if its hit rate h exceeds t_new / t_below: its own access time divided by the access time of the tier it saves you from reaching.

The remote tier sees only what the local caches missed, a filtered stream with the hot keys already removed. Its hit rate will be lower than on raw traffic, and it needs a different policy from the tier above.

Add it only for what a local cache cannot do: pool capacity across servers, hold derived results, or survive process restarts. And only if its measured hit rate on the filtered stream clears the rule.

Every commit changes the table and invalidates every cached result that reads it: w = 0.1 changes per second. Reads per version, λ / w, are now tiny and fall below the cost-to-saving ratio P / S, so entries die before they pay back. A dashboard that reads several tables is worse: its change rate is the sum of theirs.

Stop result caching for that table. Cache what does not change instead: immutable file blocks and metadata. Or maintain an incremental materialized view, keeping (sum, count) pairs so averages update from each commit's delta.

A view costs maintenance work on every commit, and it is stale between refreshes.

Cache the "not found" answers with a short TTL, in their own capacity partition with a cap, so absent keys cannot evict real articles.

For a very large ID space, put a Bloom filter of valid IDs in front, sized for a chosen false-positive rate. It turns away most bad IDs without any lookup.

When an article is created, delete its negative entry; otherwise the new article is reported missing.

A stale "not found" is the worst experience here: the content exists, and the user is told it does not.

In open table formats the manifests are immutable: cache them with no expiry. Only the pointer to the current snapshot changes; give it a short TTL, so all file listings agree on one snapshot.

Resolve columns by ID, not by name or position, so a stale schema cannot make a reader misinterpret the bytes.

Stale metadata costs differently by kind: a stale file list hides data (absence), a stale schema can give wrong answers (correctness), stale statistics give bad plans. Choose a TTL for each.

By Little's Law, 1,500 requests per second × 2 s = 3,000 duplicate queries at every expiry.

Coalescing on each server still leaves 40 recomputes across the fleet: better, but not one. Probabilistic early expiry, scaled by the measured recompute time, needs no lock and works across the fleet for a little extra work. If the homepage may be a few seconds old, serve-stale-while-revalidate, which adds up to one recompute time of staleness.

Add 10–20% jitter to every TTL, so entries filled together, for example at a deploy, do not expire together.

If the origin is already near capacity, the feedback loop can keep it down after the trigger has passed. Add load shedding.

Do not update. Write the store, then delete the key, and keep a TTL as the floor.

Two concurrent writers can leave the older value in the cache last (the update race). A reader can put the old value back after the delete, during a window as long as one read (populate-after-invalidate).

Give the editor a version token from the write and include it in the key on their later reads. They see their own save without sticky routing.

The first read after each write is a miss. Accept it, or use write-through only for keys that are reliably read right after they are written.

Leverage is 1 / (1 − 0.95) = 20: the database could see up to 20 times its provisioned load.

Do not restart all nodes at once. Snapshot and restore the cache if it supports that. Otherwise ramp traffic so that the misses, (1 − h(t)) × λ(t), stay below the origin's capacity, reading h(t) from the miss-ratio curve.

Replaying yesterday's most popular keys, if popularity shifts from day to day; and shadow traffic, if the origin has no spare capacity.

Request coalescing and a load-shedding rule. Record the cache as a dependency of the database.

Use an admission filter with a frequency sketch (W-TinyLFU). Only its small admission window, 1% of a 16 GB cache or about 160 MB, can be polluted, however long the scan. Have the crawler send a bypass hint too: it knows it will not reuse what it reads.

Keep a near-cache of the 100 hottest keys in each client, with a 5 s TTL. The skew that makes the key hot also makes the hot set small enough to fit.

Partition capacity by cost class and by tenant, with admission per partition. Pin entries only where a hard guarantee is needed, accepting that a pinned entry is no longer a cache.

Use consistent hashing with 256 virtual nodes, and reshard with the same warm-up ramp.

• Does anything write to Postgres without passing through the cache path? Then the TTL is the only freshness.
• Do cheap and expensive entries share one LRU?
• What happens during large scans, with no admission filter?
• What happens when a node is added or lost? Modulo sharding moves (N − 1) / N of the keys.
• Which keys are hot? The top key may carry several percent of all traffic.
• What does Postgres do at 1 / (1 − h) times its normal load if the cache is lost?

What the cache assumes and cannot check: no unknown writers, complete invalidation sets, popularity skew that stays the same, and TTLs that expire independently.

Consistent hashing with virtual nodes; W-TinyLFU or cost partitions; jittered TTLs; a warm-up ramp controller; a near-cache for hot keys; and the cache drawn on the dependency diagram.

A large scan (a batch job, or an analyst) evicts the hot set: LRU treats every scanned block as recently used. The hit rate falls from about 90% to near zero, so the mean moves from about 5 ms to about 50 ms. The cache's hidden assumption, that the future looks like the past, broke.

Make admission scan-resistant (do not admit blocks read sequentially, or use ARC, LRU-K or TinyLFU); give batch work a separate cache or a quota; and monitor miss latency, since misses make up about 98% of the mean.

The scan itself is not cached, which is correct for data read once.