The catch-up condition

Simulate a modelNeeds Python

Problem

After an outage, a consumer has a backlog equal to the arrival rate times the outage, and it can work it off only with the capacity left over after keeping up with new arrivals: recovery time = backlog / (M − a), where M is its maximum throughput and a the arrival rate. The spare capacity M − a shrinks as utilisation a/M approaches 100%, so recovery time grows without bound. A consumer that keeps up comfortably in normal operation can still take hours to recover from a short outage.

Check recovery time = backlog / (M - a) and see why utilization matters. Write a small simulator with a queue. Arrival rate a = 500 per second, outage of 60 seconds (nothing is processed), then processing at maximum throughput M. Try M from 525 to 2,000. Record the time after the outage until the queue is empty. Plot recovery time against utilization a/M.

The question. How long does the backlog take to clear at each throughput, and how does that grow with utilisation?

Predict

At 95% utilisation (capacity 525 against 500 arriving), how long does a one-minute outage take to recover from?

About a minuteA few minutes10 to 30 minutesOver half an hour

Run it

1Install uv once. It fetches Python and the lab's packages for you.
Terminal
```
curl -LsSf https://astral.sh/uv/install.sh | sh
```

2Get the script. It is the whole lab: read it, then download it.

the-catch-up-condition.py36 linesDownload

View the code

the-catch-up-condition.py

# /// script
# requires-python = ">=3.10"
# ///
# Copyright (c) 2026 Aayush Sharma · SPDX-License-Identifier: MIT
# https://thedataengineering.com/lab-code/LICENSE.txt
"""The catch-up condition.

Arrivals at a = 500 per second, a 60-second outage, then processing at full
throughput M: time to empty the queue for M = 550 to 2,000, against
backlog / (M - a). Recovery time explodes as a / M approaches 1.
"""
import argparse, json

p = argparse.ArgumentParser(description=__doc__.splitlines()[0])
p.add_argument("--arrival", type=float, default=500, help="a: arrivals per second.")
p.add_argument("--outage", type=float, default=60, help="Seconds with no processing.")
p.add_argument("--capacities", type=lambda s: [float(v) for v in s.split(",")], default=[525, 550, 625, 750, 1000, 2000], help="M values.")
p.add_argument("--json", help="Also write the results to this JSON file.")
args = p.parse_args()
a, T, DT = args.arrival, args.outage, 0.01

rows = []
for M in args.capacities:
    q, t = 0.0, 0.0
    while True:
        q += a * DT
        if t >= T: q = max(0.0, q - M * DT)
        t += DT
        if t > T and q <= 0: break
    rows.append({"M": M, "utilization": a / M, "recovery_s": t - T, "formula_s": a * T / (M - a)})
    print(f"M = {M:6.0f} (a/M = {a / M:4.0%}): queue empty after {t - T:7.1f} s   formula {a * T / (M - a):7.1f} s")

if args.json:
    with open(args.json, "w") as f:
        json.dump({"tables": {"capacities": rows}}, f)

3Run it in the folder you saved it to, with the defaults first. Then change a knob from the table below and run it again.
Terminal
```
uv run the-catch-up-condition.py
uv run the-catch-up-condition.py --help     # every knob
```

Knobs
Knob · default	What it changes
--arrival500	a: arrivals per second.
--outage60	Seconds with no processing.
--capacities525, 550, 625, 750, 1000, 2000	M values.

Our run

Recovery time against utilisationApple M2 · 8 cores · 8 GB

Simulatedbacklog / (M − a)

Data table

Recovery time against utilisation
Utilisation a/M	Simulated	backlog / (M − a)
0.952	1,200	1,200
0.909	600	600
0.8	240	240
0.667	120	120
0.5	60	60
0.25	20	20

macOS 26.6.2 · Python 3.13.3 · run 2026-10-20

Recovery after a 60 s outage at a = 500/s

Recovery after a 60 s outage at a = 500/s
M (/s)	a/M	Simulated	Formula
525	0.952	1,200 s	1,200 s
550	0.909	600 s	600 s
625	0.8	240 s	240 s
750	0.667	120 s	120 s
1,000	0.5	60 s	60 s
2,000	0.25	20 s	20 s

Output · uv run the-catch-up-condition.py

M =    525 (a/M =  95%): queue empty after  1200.2 s   formula  1200.0 s
M =    550 (a/M =  91%): queue empty after   600.1 s   formula   600.0 s
M =    625 (a/M =  80%): queue empty after   240.0 s   formula   240.0 s
M =    750 (a/M =  67%): queue empty after   120.0 s   formula   120.0 s
M =   1000 (a/M =  50%): queue empty after    60.0 s   formula    60.0 s
M =   2000 (a/M =  25%): queue empty after    20.0 s   formula    20.0 s

What we found

The simulation matched the formula at every throughput: 600 s at M = 550, 120 s at 750, 60 s at 1,000 and 20 s at 2,000. At M = 525, 95% utilisation, a one-minute outage took 1,200 s to recover.

Read it

Recovery time depends on spare capacity, not on capacity. A consumer sized to keep up at 95% utilisation recovers from every outage twenty times slower than it took to happen. Plan capacity for the recovery you need after the outage you expect, not for steady state, or accept that the backlog has to be shed or processed elsewhere.