Autoscaling

Why can reactive autoscaling handle trends but never spikes?

Autoscaling is a control loop with a delay:

RelationshipA delayed controller oscillates →

L = detect + decide + provision + warm up

L (loop delay): $d$ — how old the state the controller acts on is

Stated in 2 places →Principle: Local optimizations compose badly →

This is typically 30 seconds to several minutes. Control theory gives the boundary at once: a loop cannot regulate disturbances faster than its own delay.

If a burst lasts less than L, reactive autoscaling provisions capacity for a burst that has already ended, then, seeing the load fall, removes it just in time to miss the next burst. Oscillation (capacity going up and down repeatedly, "flapping") follows.

Capacity arrives after the burst and leaves before the next90 s bursts · 4 min loop delay

So state it flatly, because it is sold wrongly: reactive autoscaling handles trends, never spikes. Spikes are absorbed by three things only:

headroom you already paid for,
a queue that converts the spike into latency,
shedding (refusing some work).

Predictive scaling

Predictive scaling tries to buy back L by acting on a forecast. That trades the loop-delay problem for a forecast-error problem. The errors are asymmetric (under-provisioning hurts far more than over-provisioning), so predictive scalers should be deliberately biased high. The ones that work are.

Scale-to-zero

Scale-to-zero is the same loop with L exposed to the user as cold-start latency (the delay when the first request wakes a sleeping service). It wins when the gap between requests is much longer than the warm-up; when they are comparable, warm pools split the difference with pre-paid idle capacity bought at a fraction of full provisioning. The break-even in money is cost-mechanics'.

Autoscaling regulates nothing faster than its own loop delay. Spikes are absorbed by headroom, queue or shedding, never by scaling.