Warming and prefetch are two mechanisms, not one

When does a prefetch pay, and how fast can a cold cache be ramped without overwhelming the origin?

They are often grouped together. They share only the direction of the data movement.

Prefetch: speculation on the read path

Prefetch means fetching an item before anyone asks, hoping it will be asked for. Its economics include a cost people leave out. The value of a prefetch, and the accuracy P(hit) it needs to pay, are

where S is the saving if it hits, C_fetch is the cost of fetching, and C_evict is the cost of the entry it pushed out.

A wrong prefetch costs twice: the bandwidth, and the useful entry that it displaced. C_evict is the marginal value of the displaced entry. That is the slope of the miss-ratio curve from What makes data cacheable (the curve of miss rate against cache size). So the prefetch accuracy threshold is set by the gradient of your miss-ratio curve:

• On a plateau (a flat part), eviction costs almost nothing, so aggressive prefetch is nearly free.
• Just past a cliff (a steep drop), which is where a well-sized cache should sit, a wrong prefetch is expensive.

This leads to a result that is surprising and correct: a well-sized cache should prefetch less aggressively than a badly sized one. It explains why real prefetchers limit themselves to patterns with near-certain continuation (detecting sequential reads, fixed strides) and not anything speculative. Only near-certainty clears the threshold once C_evict is counted. A second reason for caution: prefetch uses the same bandwidth that real (demand) misses need. Aggressive prefetch can cause incast, a pile-up of many responses arriving at one receiver at once, on your own system.

Warming: survivability, not latency

Warming means filling a cache in bulk when no request is waiting. Its purpose is not lower latency. It is survivability. It exists to prevent the cold-cache collapse that the leverage ratio predicts: at h = 0.95 the origin is sized for one twentieth of the load, so sending full traffic to an empty cache gives the origin 20 times its capacity.

That makes the ramp a control problem with a set-point you can calculate. Admit traffic at a rate λ(t) such that

where h(t) is the hit rate at time t, read from the miss-ratio curve as the cache fills.

The ramp rate follows from the origin's spare capacity and the curve. It does not follow from a fixed timer, which is how ramps are almost always built, and why warm-ups either take far too long or protect nothing.

There are four warming mechanisms, and they are not equivalent:

Notice a coupling back to What makes data cacheable: every warming strategy except the ramp needs measured popularity. You cannot warm well without the instrumentation of What makes data cacheable, so the measurement work is a prerequisite for warming too, not only for sizing.

Compression: Prefetch is speculation priced against the slope of the miss-ratio curve, so a well-sized cache prefetches less. Warming is a survivability controller whose ramp rate is set by origin spare capacity. Only the traffic ramp is self-tuning.