Pinning and unpinning: where residency meets correctness
Why do pins and dirty pages make a buffer pool a correctness component, and how small is the usable pool under load?
A pin is a short-term reference count on a frame that makes it ineligible for eviction while an operator is reading or writing it. Three things follow, and they are why the buffer pool is a correctness component, not only a performance one.
First: the latch/lock distinction
| Latch (pin, buffer latch) | Lock (transaction lock) | |
|---|---|---|
| Protects | a physical structure: a frame, a page image | logical contents: a row, a range |
| Duration | one operation | one transaction |
| Deadlock handling | avoided by protocol ordering, not detected | detected or timed out |
| Visible to | the storage engine | the transaction manager, and sometimes the user |
Confusing the two produces the classic bug "my buffer pin was held across an I/O". It is a latency incident that looks like a locking incident.
Second: eviction has a dependency on the log
A dirty page cannot be written to disk before the WAL records describing its changes are durable. So the eviction path may have to force the log, on a different device, before it can free one frame. The consequence for this mechanism: the buffer pool's eviction latency is not bounded by the pool's own behaviour. Under pressure, finding a free frame can cost an fsync. That is the structural fact that makes a buffer pool categorically not a cache, and it is worth being able to say in one breath.
Third: the effective pool size is not the pool size
Under load, all three terms move the wrong way together: more concurrency means more pins, more writes mean more dirty pages, and log pressure means slower flushing. This is the "independence assumed, but false" idea in an especially unpleasant form, because the pool reports its configured size to the memory accounting of Rationing while its usable size is collapsing.
A pin leak (an operator that fails to unpin on an error path) is the chronic version: unevictable frames accumulate and the pool shrinks silently over days.
The free-frame supply, and a collector that fails like every other collector
Synchronous eviction can cost an fsync, so systems run a background writer or page cleaner that keeps a supply of clean, evictable frames ahead of demand. That is cost relocation: the write moves off the critical path onto a background thread. And it is also the failure mode of every collector: the cleaner falls behind under exactly the conditions that produce the most dirty pages. When it does, the foreground thread pays double: it searches for a victim, flushes it, and forces the log. This is structurally identical to LSM compaction debt and to flash garbage collection falling behind, and in all three the symptom looks nothing like the cause.
One-line compression: a pin makes a frame unevictable and a dirty page makes eviction depend on the log. Under pressure both grow, so the pool is smallest exactly when it is needed most.