Lifetime is phase-structured

What does general-purpose memory management charge for, and what do phase-structured lifetimes let you stop paying?

CaseConstructed case · Amara at Voltara

40-second freezes, and a process that looks like it leaks

Voltara's dashboard keeps a 60 GB cache of meter metadata inside the JVM, as hundreds of millions of small objects. Every few minutes the whole dashboard freezes for 20 to 40 seconds, then wakes up. Amara moves part of the data into off-heap buffers. The freezes get shorter, but now Dmitri sees a new oddity: the process's RSS keeps growing for days while the "heap in use" graph stays flat, so the process looks like it leaks although nothing is leaking. A week later the nightly job dies of OOM after interning every distinct request URL in a global table. And on the 2-socket servers, one machine scans 30% slower than its identical twin.

Four symptoms, one cause: someone has to decide who owns each byte, and the default owners (the garbage collector, malloc, the OS) do not know what the application knows.

How it ends →

General-purpose memory management charges for arbitrary lifetime. malloc must let you free any object at any time in any order. The garbage collector must let any reference reach any object. That one requirement generates all of its costs: per-object headers, free lists, coalescing of neighbouring free blocks, size classes, and, for a tracing collector, a traversal proportional to what survives.

But data-engineering workloads almost never need arbitrary lifetime. They have phases: a batch, a tuple, the build side of a join, a query. Inside a phase, allocations are born together and die together.

The bets differ only in what the phase is and what happens if you are wrong.

Mechanism	Freedom given up	Bought with it	Failure when the bet is wrong
Arena / region	free in arbitrary order	~50× cheaper allocation, bounded fragmentation	dangling pointer, silent
Off-heap	runtime-proved liveness	live set invisible to the collector, byte-level layout	leak or use-after-free, silent
GC tuning	nothing (you stay on the runtime)	a position on the pause/throughput trade-off	pause frequency blows up non-linearly
Arrow-style pooling	arbitrary object shape	zero-copy sharing, computable footprint	memory pinned by the slowest consumer
Interning / dictionary	value independence	repetition becomes integers	unbounded dictionary growth
NUMA binding	placement freedom	local memory bandwidth	binding fights the scheduler