A buffer pool is the path, not a shortcut

Why is a buffer-pool miss not optional, and why is eviction a transaction rather than a deletion?

CaseConstructed case · Amara at Voltara

One scan, and the hit rate drops to zero

At 10:00 am Yuki runs a full scan over 2 TB of readings to rebuild a statistics table. Before the scan, the dashboard's point lookups were served from the buffer pool with a hit rate of about 95%. During the scan the lookups slow down. After the scan finishes, the hit rate is 0% and stays near zero for 20 minutes while lookups re-read the disk. Then, in the afternoon, Amara tests the engine on a new 96-core server. Adding cores makes the buffer pool slower, not faster, even though everything is in memory. And a new engineer says, with confidence: "Why do we have a buffer pool at all? Just mmap the files and let the kernel handle it."

Three questions about the same thing: which pages are in memory, who decides, and at what cost.

How it ends →

Ownership assumed that the bytes you allocated were yours to keep. Residency is the opposite situation: the data is larger than memory by construction, permanently, and the only question is which fraction of it is present at any instant.

The difference from caching is a property that sounds pedantic and turns out to be load-bearing:

That is the whole difference. Evicting from a cache is a free decision. Evicting from a buffer pool may require writing a dirty page to disk. That may in turn require first flushing the write-ahead log, which is an fsync on a different device. Eviction is therefore a transaction with a dependency chain, not a deletion. Everything strange about buffer-pool design follows from that.

The core tension is a table of conflicts:

	The pool must…	…which conflicts with
Hit rate	keep the pages that will be referenced again	not knowing the future
Hit-path cost	record every reference, to know what is hot	every recorded reference is a write to shared state
Correctness	hold pinned and unflushed pages regardless of usefulness	the pool's usable size shrinking exactly under pressure
Sizing	be large enough to hold the working set	the marginal value of RAM falling steeply while it costs money

Rows 1 and 2 fight each other directly. A better replacement policy needs more information about references. Collecting that information costs shared-memory writes on the fastest path in the system. Most of the design history of buffer pools is not a search for a better policy. It is a search for a policy whose bookkeeping is cheap enough to be worth its own hit rate. Hold that thought: it explains why the theoretically best algorithms are the ones production systems do not run.