An autoscaler that arrives too late

Simulate a modelNeeds Python

Problem

A reactive autoscaler adds capacity after it sees demand, and capacity arrives after a provisioning delay. A burst shorter than that delay is over before the new capacity is ready, so the scaler serves none of it, and the capacity it adds arrives into the quiet afterwards. Only capacity that is already there, headroom, serves a short burst, and headroom is paid for all the time.

Model a reactive autoscaler whose capacity at time t equals the demand it saw 240 seconds earlier. Feed it a demand curve with two 90-second bursts from 100 to 300, and count the demand that goes unserved. Then add permanent headroom (30%, 100%, 200%) and compare the unserved demand and the idle capacity you pay for.

The question. How much of the bursts goes unserved at each headroom, and how much idle capacity does the headroom cost?

Predict

With 30% headroom and a 4-minute provisioning delay, what share of the bursts goes unserved?

NoneUnder halfMostAll of them

Run it

1Install uv once. It fetches Python and the lab's packages for you.
Terminal
```
curl -LsSf https://astral.sh/uv/install.sh | sh
```

2Get the script. It is the whole lab: read it, then download it.

an-autoscaler-that-arrives-too-late.py41 linesDownload

View the code

an-autoscaler-that-arrives-too-late.py

# /// script
# requires-python = ">=3.10"
# ///
# Copyright (c) 2026 Aayush Sharma · SPDX-License-Identifier: MIT
# https://thedataengineering.com/lab-code/LICENSE.txt
"""An autoscaler that arrives too late.

A reactive autoscaler whose capacity at time t equals the demand it saw DELAY
seconds earlier (never below a floor of 100 × (1 + headroom)), against a
demand curve with two 90-second bursts. Unserved and idle demand-seconds for
headroom from 0% to 200%.
"""
import argparse, json

p = argparse.ArgumentParser(description=__doc__.splitlines()[0])
p.add_argument("--delay", type=int, default=240, help="Seconds between seeing demand and having the capacity.")
p.add_argument("--headrooms", default="0,0.3,1.0,2.0", help="Permanent headroom fractions.")
p.add_argument("--json", help="Also write the results to this JSON file.")
args = p.parse_args()

demand = [100] * 100 + [300] * 90 + [100] * 400 + [300] * 90 + [100] * 300   # one value per second
def evaluate(headroom):
    unserved = idle = 0; cap_series = []
    for t, d in enumerate(demand):
        seen = demand[max(0, t - args.delay)]                              # what the scaler saw DELAY seconds ago
        cap = max(seen, 100 * (1 + headroom))                              # reactive capacity, plus a floor
        unserved += max(0, d - cap); idle += max(0, cap - d); cap_series.append(cap)
    return unserved, idle, cap_series
rows, timeline = [], []
burst = sum(max(0, d - 100) for d in demand)
for h in map(float, args.headrooms.split(",")):
    u, i, caps = evaluate(h)
    rows.append({"headroom_pct": h * 100, "unserved": u, "unserved_pct_of_burst": u / burst * 100, "idle": i})
    if h in (0.0, 2.0): timeline += [{"second": t, "series": f"capacity, headroom {h:.0%}", "value": c} for t, c in enumerate(caps) if t % 5 == 0]
timeline += [{"second": t, "series": "demand", "value": d} for t, d in enumerate(demand) if t % 5 == 0]
for r in rows: print(f"headroom={r['headroom_pct']:4.0f}%  unserved={r['unserved']:6.0f} ({r['unserved_pct_of_burst']:5.1f}% of the bursts)  idle={r['idle']:7.0f}   (demand-seconds)")

if args.json:
    with open(args.json, "w") as f:
        json.dump({"tables": {"headroom": rows, "timeline": timeline}}, f)

3Run it in the folder you saved it to, with the defaults first. Then change a knob from the table below and run it again.
Terminal
```
uv run an-autoscaler-that-arrives-too-late.py
uv run an-autoscaler-that-arrives-too-late.py --help     # every knob
```

Knobs
Knob · default	What it changes
--delay240	Seconds between seeing demand and having the capacity.
--headrooms0, 0.3, 1.0, 2.0	Permanent headroom fractions.

Our run

Demand and capacityApple M2 · 8 cores · 8 GB

capacity, headroom 0%capacity, headroom 200%demand

Data table

Demand and capacity
Second	series	Requests/s
0	capacity, headroom 0%	100
5	capacity, headroom 0%	100
10	capacity, headroom 0%	100
15	capacity, headroom 0%	100
20	capacity, headroom 0%	100
25	capacity, headroom 0%	100
30	capacity, headroom 0%	100
35	capacity, headroom 0%	100
40	capacity, headroom 0%	100
45	capacity, headroom 0%	100
50	capacity, headroom 0%	100
55	capacity, headroom 0%	100
60	capacity, headroom 0%	100
65	capacity, headroom 0%	100
70	capacity, headroom 0%	100
75	capacity, headroom 0%	100
80	capacity, headroom 0%	100
85	capacity, headroom 0%	100
90	capacity, headroom 0%	100
95	capacity, headroom 0%	100
100	capacity, headroom 0%	100
105	capacity, headroom 0%	100
110	capacity, headroom 0%	100
115	capacity, headroom 0%	100
120	capacity, headroom 0%	100
125	capacity, headroom 0%	100
130	capacity, headroom 0%	100
135	capacity, headroom 0%	100
140	capacity, headroom 0%	100
145	capacity, headroom 0%	100
150	capacity, headroom 0%	100
155	capacity, headroom 0%	100
160	capacity, headroom 0%	100
165	capacity, headroom 0%	100
170	capacity, headroom 0%	100
175	capacity, headroom 0%	100
180	capacity, headroom 0%	100
185	capacity, headroom 0%	100
190	capacity, headroom 0%	100
195	capacity, headroom 0%	100
200	capacity, headroom 0%	100
205	capacity, headroom 0%	100
210	capacity, headroom 0%	100
215	capacity, headroom 0%	100
220	capacity, headroom 0%	100
225	capacity, headroom 0%	100
230	capacity, headroom 0%	100
235	capacity, headroom 0%	100
240	capacity, headroom 0%	100
245	capacity, headroom 0%	100
250	capacity, headroom 0%	100
255	capacity, headroom 0%	100
260	capacity, headroom 0%	100
265	capacity, headroom 0%	100
270	capacity, headroom 0%	100
275	capacity, headroom 0%	100
280	capacity, headroom 0%	100
285	capacity, headroom 0%	100
290	capacity, headroom 0%	100
295	capacity, headroom 0%	100
300	capacity, headroom 0%	100
305	capacity, headroom 0%	100
310	capacity, headroom 0%	100
315	capacity, headroom 0%	100
320	capacity, headroom 0%	100
325	capacity, headroom 0%	100
330	capacity, headroom 0%	100
335	capacity, headroom 0%	100
340	capacity, headroom 0%	300
345	capacity, headroom 0%	300
350	capacity, headroom 0%	300
355	capacity, headroom 0%	300
360	capacity, headroom 0%	300
365	capacity, headroom 0%	300
370	capacity, headroom 0%	300
375	capacity, headroom 0%	300
380	capacity, headroom 0%	300
385	capacity, headroom 0%	300
390	capacity, headroom 0%	300
395	capacity, headroom 0%	300
400	capacity, headroom 0%	300
405	capacity, headroom 0%	300
410	capacity, headroom 0%	300
415	capacity, headroom 0%	300
420	capacity, headroom 0%	300
425	capacity, headroom 0%	300
430	capacity, headroom 0%	100
435	capacity, headroom 0%	100
440	capacity, headroom 0%	100
445	capacity, headroom 0%	100
450	capacity, headroom 0%	100
455	capacity, headroom 0%	100
460	capacity, headroom 0%	100
465	capacity, headroom 0%	100
470	capacity, headroom 0%	100
475	capacity, headroom 0%	100
480	capacity, headroom 0%	100
485	capacity, headroom 0%	100
490	capacity, headroom 0%	100
495	capacity, headroom 0%	100
500	capacity, headroom 0%	100
505	capacity, headroom 0%	100
510	capacity, headroom 0%	100
515	capacity, headroom 0%	100
520	capacity, headroom 0%	100
525	capacity, headroom 0%	100
530	capacity, headroom 0%	100
535	capacity, headroom 0%	100
540	capacity, headroom 0%	100
545	capacity, headroom 0%	100
550	capacity, headroom 0%	100
555	capacity, headroom 0%	100
560	capacity, headroom 0%	100
565	capacity, headroom 0%	100
570	capacity, headroom 0%	100
575	capacity, headroom 0%	100
580	capacity, headroom 0%	100
585	capacity, headroom 0%	100
590	capacity, headroom 0%	100
595	capacity, headroom 0%	100
600	capacity, headroom 0%	100
605	capacity, headroom 0%	100
610	capacity, headroom 0%	100
615	capacity, headroom 0%	100
620	capacity, headroom 0%	100
625	capacity, headroom 0%	100
630	capacity, headroom 0%	100
635	capacity, headroom 0%	100
640	capacity, headroom 0%	100
645	capacity, headroom 0%	100
650	capacity, headroom 0%	100
655	capacity, headroom 0%	100
660	capacity, headroom 0%	100
665	capacity, headroom 0%	100
670	capacity, headroom 0%	100
675	capacity, headroom 0%	100
680	capacity, headroom 0%	100
685	capacity, headroom 0%	100
690	capacity, headroom 0%	100
695	capacity, headroom 0%	100
700	capacity, headroom 0%	100
705	capacity, headroom 0%	100
710	capacity, headroom 0%	100
715	capacity, headroom 0%	100
720	capacity, headroom 0%	100
725	capacity, headroom 0%	100
730	capacity, headroom 0%	100
735	capacity, headroom 0%	100
740	capacity, headroom 0%	100
745	capacity, headroom 0%	100
750	capacity, headroom 0%	100
755	capacity, headroom 0%	100
760	capacity, headroom 0%	100
765	capacity, headroom 0%	100
770	capacity, headroom 0%	100
775	capacity, headroom 0%	100
780	capacity, headroom 0%	100
785	capacity, headroom 0%	100
790	capacity, headroom 0%	100
795	capacity, headroom 0%	100
800	capacity, headroom 0%	100
805	capacity, headroom 0%	100
810	capacity, headroom 0%	100
815	capacity, headroom 0%	100
820	capacity, headroom 0%	100
825	capacity, headroom 0%	100
830	capacity, headroom 0%	300
835	capacity, headroom 0%	300
840	capacity, headroom 0%	300
845	capacity, headroom 0%	300
850	capacity, headroom 0%	300
855	capacity, headroom 0%	300
860	capacity, headroom 0%	300
865	capacity, headroom 0%	300
870	capacity, headroom 0%	300
875	capacity, headroom 0%	300
880	capacity, headroom 0%	300
885	capacity, headroom 0%	300
890	capacity, headroom 0%	300
895	capacity, headroom 0%	300
900	capacity, headroom 0%	300
905	capacity, headroom 0%	300
910	capacity, headroom 0%	300
915	capacity, headroom 0%	300
920	capacity, headroom 0%	100
925	capacity, headroom 0%	100
930	capacity, headroom 0%	100
935	capacity, headroom 0%	100
940	capacity, headroom 0%	100
945	capacity, headroom 0%	100
950	capacity, headroom 0%	100
955	capacity, headroom 0%	100
960	capacity, headroom 0%	100
965	capacity, headroom 0%	100
970	capacity, headroom 0%	100
975	capacity, headroom 0%	100
0	capacity, headroom 200%	300
5	capacity, headroom 200%	300
10	capacity, headroom 200%	300
15	capacity, headroom 200%	300
20	capacity, headroom 200%	300
25	capacity, headroom 200%	300
30	capacity, headroom 200%	300
35	capacity, headroom 200%	300
40	capacity, headroom 200%	300
45	capacity, headroom 200%	300
50	capacity, headroom 200%	300
55	capacity, headroom 200%	300
60	capacity, headroom 200%	300
65	capacity, headroom 200%	300
70	capacity, headroom 200%	300
75	capacity, headroom 200%	300
80	capacity, headroom 200%	300
85	capacity, headroom 200%	300
90	capacity, headroom 200%	300
95	capacity, headroom 200%	300
100	capacity, headroom 200%	300
105	capacity, headroom 200%	300
110	capacity, headroom 200%	300
115	capacity, headroom 200%	300
120	capacity, headroom 200%	300
125	capacity, headroom 200%	300
130	capacity, headroom 200%	300
135	capacity, headroom 200%	300
140	capacity, headroom 200%	300
145	capacity, headroom 200%	300
150	capacity, headroom 200%	300
155	capacity, headroom 200%	300
160	capacity, headroom 200%	300
165	capacity, headroom 200%	300
170	capacity, headroom 200%	300
175	capacity, headroom 200%	300
180	capacity, headroom 200%	300
185	capacity, headroom 200%	300
190	capacity, headroom 200%	300
195	capacity, headroom 200%	300
200	capacity, headroom 200%	300
205	capacity, headroom 200%	300
210	capacity, headroom 200%	300
215	capacity, headroom 200%	300
220	capacity, headroom 200%	300
225	capacity, headroom 200%	300
230	capacity, headroom 200%	300
235	capacity, headroom 200%	300
240	capacity, headroom 200%	300
245	capacity, headroom 200%	300
250	capacity, headroom 200%	300
255	capacity, headroom 200%	300
260	capacity, headroom 200%	300
265	capacity, headroom 200%	300
270	capacity, headroom 200%	300
275	capacity, headroom 200%	300
280	capacity, headroom 200%	300
285	capacity, headroom 200%	300
290	capacity, headroom 200%	300
295	capacity, headroom 200%	300
300	capacity, headroom 200%	300
305	capacity, headroom 200%	300
310	capacity, headroom 200%	300
315	capacity, headroom 200%	300
320	capacity, headroom 200%	300
325	capacity, headroom 200%	300
330	capacity, headroom 200%	300
335	capacity, headroom 200%	300
340	capacity, headroom 200%	300
345	capacity, headroom 200%	300
350	capacity, headroom 200%	300
355	capacity, headroom 200%	300
360	capacity, headroom 200%	300
365	capacity, headroom 200%	300
370	capacity, headroom 200%	300
375	capacity, headroom 200%	300
380	capacity, headroom 200%	300
385	capacity, headroom 200%	300
390	capacity, headroom 200%	300
395	capacity, headroom 200%	300
400	capacity, headroom 200%	300
405	capacity, headroom 200%	300
410	capacity, headroom 200%	300
415	capacity, headroom 200%	300
420	capacity, headroom 200%	300
425	capacity, headroom 200%	300
430	capacity, headroom 200%	300
435	capacity, headroom 200%	300
440	capacity, headroom 200%	300
445	capacity, headroom 200%	300
450	capacity, headroom 200%	300
455	capacity, headroom 200%	300
460	capacity, headroom 200%	300
465	capacity, headroom 200%	300
470	capacity, headroom 200%	300
475	capacity, headroom 200%	300
480	capacity, headroom 200%	300
485	capacity, headroom 200%	300
490	capacity, headroom 200%	300
495	capacity, headroom 200%	300
500	capacity, headroom 200%	300
505	capacity, headroom 200%	300
510	capacity, headroom 200%	300
515	capacity, headroom 200%	300
520	capacity, headroom 200%	300
525	capacity, headroom 200%	300
530	capacity, headroom 200%	300
535	capacity, headroom 200%	300
540	capacity, headroom 200%	300
545	capacity, headroom 200%	300
550	capacity, headroom 200%	300
555	capacity, headroom 200%	300
560	capacity, headroom 200%	300
565	capacity, headroom 200%	300
570	capacity, headroom 200%	300
575	capacity, headroom 200%	300
580	capacity, headroom 200%	300
585	capacity, headroom 200%	300
590	capacity, headroom 200%	300
595	capacity, headroom 200%	300
600	capacity, headroom 200%	300
605	capacity, headroom 200%	300
610	capacity, headroom 200%	300
615	capacity, headroom 200%	300
620	capacity, headroom 200%	300
625	capacity, headroom 200%	300
630	capacity, headroom 200%	300
635	capacity, headroom 200%	300
640	capacity, headroom 200%	300
645	capacity, headroom 200%	300
650	capacity, headroom 200%	300
655	capacity, headroom 200%	300
660	capacity, headroom 200%	300
665	capacity, headroom 200%	300
670	capacity, headroom 200%	300
675	capacity, headroom 200%	300
680	capacity, headroom 200%	300
685	capacity, headroom 200%	300
690	capacity, headroom 200%	300
695	capacity, headroom 200%	300
700	capacity, headroom 200%	300
705	capacity, headroom 200%	300
710	capacity, headroom 200%	300
715	capacity, headroom 200%	300
720	capacity, headroom 200%	300
725	capacity, headroom 200%	300
730	capacity, headroom 200%	300
735	capacity, headroom 200%	300
740	capacity, headroom 200%	300
745	capacity, headroom 200%	300
750	capacity, headroom 200%	300
755	capacity, headroom 200%	300
760	capacity, headroom 200%	300
765	capacity, headroom 200%	300
770	capacity, headroom 200%	300
775	capacity, headroom 200%	300
780	capacity, headroom 200%	300
785	capacity, headroom 200%	300
790	capacity, headroom 200%	300
795	capacity, headroom 200%	300
800	capacity, headroom 200%	300
805	capacity, headroom 200%	300
810	capacity, headroom 200%	300
815	capacity, headroom 200%	300
820	capacity, headroom 200%	300
825	capacity, headroom 200%	300
830	capacity, headroom 200%	300
835	capacity, headroom 200%	300
840	capacity, headroom 200%	300
845	capacity, headroom 200%	300
850	capacity, headroom 200%	300
855	capacity, headroom 200%	300
860	capacity, headroom 200%	300
865	capacity, headroom 200%	300
870	capacity, headroom 200%	300
875	capacity, headroom 200%	300
880	capacity, headroom 200%	300
885	capacity, headroom 200%	300
890	capacity, headroom 200%	300
895	capacity, headroom 200%	300
900	capacity, headroom 200%	300
905	capacity, headroom 200%	300
910	capacity, headroom 200%	300
915	capacity, headroom 200%	300
920	capacity, headroom 200%	300
925	capacity, headroom 200%	300
930	capacity, headroom 200%	300
935	capacity, headroom 200%	300
940	capacity, headroom 200%	300
945	capacity, headroom 200%	300
950	capacity, headroom 200%	300
955	capacity, headroom 200%	300
960	capacity, headroom 200%	300
965	capacity, headroom 200%	300
970	capacity, headroom 200%	300
975	capacity, headroom 200%	300
0	demand	100
5	demand	100
10	demand	100
15	demand	100
20	demand	100
25	demand	100
30	demand	100
35	demand	100
40	demand	100
45	demand	100
50	demand	100
55	demand	100
60	demand	100
65	demand	100
70	demand	100
75	demand	100
80	demand	100
85	demand	100
90	demand	100
95	demand	100
100	demand	300
105	demand	300
110	demand	300
115	demand	300
120	demand	300
125	demand	300
130	demand	300
135	demand	300
140	demand	300
145	demand	300
150	demand	300
155	demand	300
160	demand	300
165	demand	300
170	demand	300
175	demand	300
180	demand	300
185	demand	300
190	demand	100
195	demand	100
200	demand	100
205	demand	100
210	demand	100
215	demand	100
220	demand	100
225	demand	100
230	demand	100
235	demand	100
240	demand	100
245	demand	100
250	demand	100
255	demand	100
260	demand	100
265	demand	100
270	demand	100
275	demand	100
280	demand	100
285	demand	100
290	demand	100
295	demand	100
300	demand	100
305	demand	100
310	demand	100
315	demand	100
320	demand	100
325	demand	100
330	demand	100
335	demand	100
340	demand	100
345	demand	100
350	demand	100
355	demand	100
360	demand	100
365	demand	100
370	demand	100
375	demand	100
380	demand	100
385	demand	100
390	demand	100
395	demand	100
400	demand	100
405	demand	100
410	demand	100
415	demand	100
420	demand	100
425	demand	100
430	demand	100
435	demand	100
440	demand	100
445	demand	100
450	demand	100
455	demand	100
460	demand	100
465	demand	100
470	demand	100
475	demand	100
480	demand	100
485	demand	100
490	demand	100
495	demand	100
500	demand	100
505	demand	100
510	demand	100
515	demand	100
520	demand	100
525	demand	100
530	demand	100
535	demand	100
540	demand	100
545	demand	100
550	demand	100
555	demand	100
560	demand	100
565	demand	100
570	demand	100
575	demand	100
580	demand	100
585	demand	100
590	demand	300
595	demand	300
600	demand	300
605	demand	300
610	demand	300
615	demand	300
620	demand	300
625	demand	300
630	demand	300
635	demand	300
640	demand	300
645	demand	300
650	demand	300
655	demand	300
660	demand	300
665	demand	300
670	demand	300
675	demand	300
680	demand	100
685	demand	100
690	demand	100
695	demand	100
700	demand	100
705	demand	100
710	demand	100
715	demand	100
720	demand	100
725	demand	100
730	demand	100
735	demand	100
740	demand	100
745	demand	100
750	demand	100
755	demand	100
760	demand	100
765	demand	100
770	demand	100
775	demand	100
780	demand	100
785	demand	100
790	demand	100
795	demand	100
800	demand	100
805	demand	100
810	demand	100
815	demand	100
820	demand	100
825	demand	100
830	demand	100
835	demand	100
840	demand	100
845	demand	100
850	demand	100
855	demand	100
860	demand	100
865	demand	100
870	demand	100
875	demand	100
880	demand	100
885	demand	100
890	demand	100
895	demand	100
900	demand	100
905	demand	100
910	demand	100
915	demand	100
920	demand	100
925	demand	100
930	demand	100
935	demand	100
940	demand	100
945	demand	100
950	demand	100
955	demand	100
960	demand	100
965	demand	100
970	demand	100
975	demand	100

macOS 26.6.2 · Python 3.13.3 · run 2026-10-20

Unserved and idle demand-seconds

Unserved and idle demand-seconds
Headroom	Unserved	Of the bursts	Idle
0%	36K	100%	36K
30%	30.6K	85%	54.6K
100%	18K	50%	98K
200%	0	0%	160K

Output · uv run an-autoscaler-that-arrives-too-late.py

headroom=   0%  unserved= 36000 (100.0% of the bursts)  idle=  36000   (demand-seconds)
headroom=  30%  unserved= 30600 ( 85.0% of the bursts)  idle=  54600   (demand-seconds)
headroom= 100%  unserved= 18000 ( 50.0% of the bursts)  idle=  98000   (demand-seconds)
headroom= 200%  unserved=     0 (  0.0% of the bursts)  idle= 160000   (demand-seconds)

What we found

With no headroom, 100% of the bursts went unserved, and the capacity added for them arrived afterwards as 36K idle demand-seconds. 30% headroom still left 85% unserved; 100% left 50%; only 200% served the bursts, at 160K idle demand-seconds.

Read it

A scaler cannot react to a burst shorter than its delay; for those, the choice is headroom (pay for idle), prediction (scale before the burst), or shedding (decide who goes unserved). A queue or load shedding changes how the unserved part feels, not whether it exists.