Choose who owns the memory
A data engine allocates memory for the data it processes: hash tables, batches of rows, string values. Either a garbage collector owns that memory and finds what is dead, or your code owns it and releases it at points you choose. Taking ownership saves the collector's work, but every allocation then needs a known moment of release. Who should own this memory?
You will end with who owns each kind of allocation, when it is released, and what sets your peak.
Decide
Is the live data large, uniform and long-lived?
Large hash tables and batches that survive for a whole query, as opposed to many small objects that die young.
Why
A tracing collector's cost grows with what survives, not with the garbage. Small objects that die young are almost free to collect, so managing them yourself is a net loss. Large, long-lived data is what the collector pays for again and again.
Do the allocations within one unit of work all die together?
Everything built for one query, one batch or one task, released when it ends.
Why
If everything has the same lifetime, you can hand out memory by bumping a pointer and free it all at once by resetting the arena: no per-object bookkeeping.
Can you compute the size before allocating?
Fixed-width columns in a batch of known row count: yes. Strings of unknown length: no.
Are the values heavily repeated?
Country codes, status strings, a few thousand distinct values across millions of rows.
Why
Storing each distinct value once and referring to it by a small code saves memory in proportion to the repetition.
Where you can land
Leave it to the garbage collector
Use it when the live data is small, varied or short-lived.
Then: size the young generation so objects die before they are promoted, and stop. Extra complexity will not pay on this side of the crossover.
Arena per unit of work
Use it when allocations share the lifetime of a query, batch or task.
Before you build it
Name the reset point out loud
State exactly when the arena is reset: the end of which query, batch or task. If you cannot name it: you do not have an arena, you have a leak with a fast allocator.
Typed pool with fixed alignment
Use it when sizes are known before allocating.
Buffers of known sizes, aligned for vector instructions, accounted per pool and reused.
What it costs: accounting, and reference counting if buffers are shared without copying.
Dictionary
Use it when values repeat heavily.
Before you build it
Decide its scope and its cap now
Choose whether a dictionary lives for one batch, one pipeline or the whole process, and how large it may grow. Choosing the encoding without the scope is how the leak ships: a global dictionary over unbounded values never stops growing.
Growable buffers with a cap
Use it when sizes are unknown and values do not repeat.
What it costs: a growth strategy (doubling, chunking) and an explicit maximum, with a decision about what happens at it.
Whichever you land on
Know who touches each buffer first
On machines with several memory nodes (NUMA), memory is placed on the node of the thread that first writes it. Whoever first touches a buffer decides its placement, whether or not you chose that.
Find who holds references longest
Any component that keeps a reference longer than the pipeline (a cache, a library, a result buffer) sets your real peak, not your allocator. That peak is the number memory rationing must reserve for.