Skip to content
The Data EngineeringThe Data EngineeringBeta
About
Main content

The partition of RAM

How should RAM be split between buffer pool, working memory, page cache and overhead, and when are you double-buffering?

A node's RAM has four claimants, and the split is zero-sum:

DefinitionThe partition of RAM →
the buffer pool
concurrency × per-query working memory
the operating system page cache
everything else the process and OS need

where

  • B = the (the database's own cache of file pages),
  • c·g = concurrency × per-query working memory (c queries, each with g bytes of hash tables, sort buffers and so on),
  • P = the OS (the kernel's own cache of file pages),
  • O = runtime overhead: thread stacks, connection buffers, metadata, the JVM itself.
Four claimants on 256 GBworked example

The structurally important fact is this: B and P both hold copies of file pages, which can always be re-read from disk. Working memory holds state that exists nowhere else. So the sizing decision is two decisions, not one: first split cached data from working state, then decide who caches.

Double buffering and O_DIRECT

If you read through ordinary (buffered) I/O into your own pool, the same page exists twice: once in the kernel's page cache and once in yours. Your effective cache capacity is somewhere between B and B + P, depending on the access pattern. That, not raw speed, is the primary reason O_DIRECT exists in database engines. O_DIRECT is a flag that makes reads and writes skip the kernel's page cache. It removes the duplicate and gives the eviction decision back to the engine. (Direct versus buffered I/O is an access-method topic; here it is an allocation decision.)

The counter-example: Kafka

Kafka does the opposite on purpose: no application-level data cache, everything through the OS page cache. Its access pattern is a log whose tail is read shortly after it is written. For that pattern the kernel's recency heuristic is exactly right and the application has no extra information to add.

This refines the claim from Memory is a space resource: the application takes the policy back when it has information the layer below lacks, not merely because it is higher up. Kafka has none, so it does not.

One-line compression: RAM has four claimants, only two of which are substitutes. If you cannot say which two, you are double-buffering.