GC tuning versus manual management

How does headroom set collector cost, and where is the ceiling of tuning?

Suppose you have not gone off-heap. What does tuning buy, and where is its ceiling?

The one load-bearing relationship

Collector CPU overhead falls roughly as

where H is the heap size and L is the live set. Headroom (H − L) is what you are buying.

(Strictly, it is doubling the headroom H − L, not the heap, that halves collection frequency. In the table above, going from 40 GB to 80 GB triples the headroom, from 20 GB to 60 GB, and cuts the relative overhead from 1 to 0.33.)

This one formula explains why both "heap too small" and "heap too large" produce pauses, by different routes:

• Too small: H/L → 1, so collection frequency diverges.
• Too large: each traversal takes longer and touches more cold memory, so pause duration and page-cache damage rise.

The knee is not a matter of taste. It is the shape of that curve.

The three knobs, described as what they are

• Heap size buys headroom, the denominator above.
• Young-generation sizing buys the death-before-promotion property. Premature promotion is the classic pathology: a hash table built through a young generation that is too small gets copied into the old generation, where collecting it is expensive. You paid for the copy and for the expensive collection, for an object that would otherwise have died for free.
• Collector choice is an explicit position on the latency/throughput trade-off. Throughput collectors maximize work per CPU-second. Low-pause collectors buy bounded pauses with concurrent relocation and program barriers, and spend throughput to do it.

The ceiling

No tuning changes the fact that the collector's cost is a function of the live set and that the collector cannot see your semantics. There are only two stable positions:

1. When the live set is dominated by one structure that you can describe in bytes rather than in objects, that structure should leave the heap.
2. When the live set is small and the problem is allocation churn, tune the young generation and stop.

Everything between the two is a system that will page you at night.

For a Spark 2.4 / JDK 8 environment the low-pause escape (ZGC-class collectors) is not available. The usable levers are the choice among Parallel, CMS and G1, young-generation sizing, and Tungsten's off-heap mode. So the crossover to "take it off-heap" arrives at a smaller live set than on a modern runtime. That is a real constraint on the decision, not a footnote.

Contention

A collector is a contended resource in the 1/(1−ρ) sense, where ρ is the allocation rate divided by the collection capacity. Pause frequency, and the latency it causes, rises non-linearly as ρ approaches 1. It is the same curve as hash-table load factor, a simple queue, and scalability laws. (What happens past the knee, where pauses slow the program and the backlog grows, is the GC death spiral, which is a failure-mode topic. Here we only need the curve.)

One-line compression: GC cost ≈ live set ÷ headroom. Tuning moves you along that curve; off-heap changes which term you are standing on.