Memory cannot be time-sliced
Why can memory not be shared by taking turns, and what two moves does that leave?
In many textbooks, memory is either background physics (how fast a cache line moves) or a free variable (an algorithm analysed "with M bytes of memory"). In this mechanism M stops being a parameter. It becomes a contested, allocated quantity with an owner.
The nature of the resource is different from the others.
- CPU is time-sliced. If two queries want the whole machine, each gets half of the cycles and both finish, a little slower.
- Disk bandwidth and network bandwidth are shared the same way: each user gets a share of the time.
- Memory is not. A hash join does not consume memory at a rate. It holds memory. It holds it for the whole duration of its build phase, whether you like it or not. Giving the join half of the memory for twice as long does not work.
This gives the sentence that generates this whole mechanism:
(Preempt means to take a resource away from its current holder in the middle of its use.)
The two moves give the whole list of mechanisms:
- Rationed before the work starts: memory grants and reservations, admission control by footprint (how many bytes a query will need), arenas, pool sizing.
- Reformulated when the ration runs out: spilling, pressure signals, operator cooperation, eviction, degradation.