What it costs to take a resource back

Why does memory need an admission gate while CPU and disk are scheduled, and what does the gate force you to estimate?

CaseConstructed case · Amara at Voltara

A tiny lookup killed, a giant join spared

It is Monday, 9:00 am. The billing batch is finishing, the analysts open their dashboards, and Yuki starts two large joins. Within three minutes, twelve queries die with "out of memory". Dmitri opens the incident. Two facts make no sense to him:

A tiny dashboard lookup, a query that needs a few kilobytes, was one of the victims. A giant join that holds over 100 GB survived.
On the other Meterbase cluster, which uses a stricter rule, the opposite happens. Queries wait in a queue for minutes while the memory graph shows 60% of RAM idle.

And a third, quieter problem: one analyst's custom function (a UDF, a user-defined function, which is code written by a user and run inside the engine) seems to use memory that no Meterbase counter shows. Nobody can say who is holding it.

Dmitri asks: "Is the memory manager broken?" It is not broken. It is making the trade-off that every memory manager must make, and it is making it blindly. To see how, we first ask why memory needs a gate while CPU and disk do not.

How it ends →

What does it cost to take a resource back?

Ownership and Residency were about one consumer using memory well. This article is about arbitration: many consumers share one pool, and someone must decide who gets what. Everything in it follows from one property. Take any machine resource and ask: what does it cost to take it back from a holder in the middle of use?

Resource	Preemption cost (cost of taking it back)	Control mechanism
CPU	one context switch, ~1–5 µs	continuous scheduling
Disk bandwidth	finish the current I/O	queue and reorder
Network	one packet	shape and pace
Memory capacity	everything the holder has built so far	?

(Preemption means taking a resource away from a holder who has not finished with it.)

For memory, the state is the work. Imagine a hash join that has spent four minutes building a 40 GB hash table. If you take 40 GB away from it, you do not slow the join down. You destroy it.

An admission gate is a check at the door: "may this query start, and with how much memory?" And an admission gate always brings an estimation problem with it, because you must know the demand before the demand exists. This one fact generates the whole article. There are three mechanisms (grants, admission, pressure signalling) and one problem under all of them: the estimate is wrong, and it is wrong in a known direction.

The core tension

You can run the gate in two ways:

	Reserve conservatively	Over-commit
Buys	isolation, predictable latency, failures you can attribute	utilization, higher concurrency
Costs	memory reserved and never used; queries queued that would have fit	correlated OOM when peaks coincide
Assumes	nothing	that the peaks of different queries are independent
Who pays when wrong	the queued query, visibly	an arbitrary query, invisibly

There is no third option. Where you stand on this line is a statement about how strongly you believe your workload's peaks are correlated (happen at the same time). The recurring problem here is that independence is assumed but false: components are assumed to peak at different times, and they usually peak together. Monday at 9 am is the example.