Retry storm: no jitter versus full jitter

Simulate a modelNeeds Python

Problem

When a shared dependency fails, every client sees the failure at the same moment and starts its retry schedule at the same moment. With plain exponential backoff, the schedules stay identical: all clients come back together, find the server at capacity, back off by the same amount, and come back together again. The retries arrive in waves, and each wave is mostly rejected. Jitter randomises each wait so that retries spread over time; full jitter draws the whole wait at random, equal jitter keeps half of it fixed.

Measure how jitter changes load and completion time after a shared failure. Simulate 400 clients. A server handles at most 50 requests per second and is down for the first 2 seconds. Each client retries with exponential backoff (base 0.1 s, cap 10 s, doubling). Run it three ways: no jitter, "equal" jitter, and full jitter (random.uniform(0, min(cap, base*2**n))). Count the total number of attempts too.

The question. How long until every client is served, and how many attempts does each backoff policy make?

Predict

Which backoff policy makes the fewest attempts in total?

No jitterEqual jitterFull jitter

Run it

1Install uv once. It fetches Python and the lab's packages for you.
Terminal
```
curl -LsSf https://astral.sh/uv/install.sh | sh
```

2Get the script. It is the whole lab: read it, then download it.

retry-storm-no-jitter-versus-full.py46 linesDownload

View the code

retry-storm-no-jitter-versus-full.py

# /// script
# requires-python = ">=3.10"
# ///
# Copyright (c) 2026 Aayush Sharma · SPDX-License-Identifier: MIT
# https://thedataengineering.com/lab-code/LICENSE.txt
"""Retry storm: no jitter, equal jitter, full jitter.

400 clients against a server that serves at most 50 requests per second and
is down for the first 2 seconds; exponential backoff (base 0.1 s, cap 10 s)
with no jitter, equal jitter and full jitter. Completion time, the busiest
second, and total attempts.
"""
import argparse, heapq, json, random

p = argparse.ArgumentParser(description=__doc__.splitlines()[0])
p.add_argument("--clients", type=int, default=400, help="Clients.")
p.add_argument("--capacity", type=int, default=50, help="Requests served per second.")
p.add_argument("--down", type=float, default=2.0, help="Seconds the server is down at the start.")
p.add_argument("--seed", type=int, default=0, help="Random seed.")
p.add_argument("--json", help="Also write the results to this JSON file.")
args = p.parse_args()

def run(jitter):
    random.seed(args.seed)
    events = [(random.uniform(0, 0.01), i, 0) for i in range(args.clients)]
    heapq.heapify(events)
    served, attempts, done_at, arrivals = {}, 0, 0.0, {}
    while events:
        t, c, n = heapq.heappop(events); attempts += 1
        sec = int(t); arrivals[sec] = arrivals.get(sec, 0) + 1
        if t < args.down or served.get(sec, 0) >= args.capacity:
            wait = min(10, 0.1 * 2 ** n)
            if jitter == "full": wait = random.uniform(0, wait)
            elif jitter == "equal": wait = wait / 2 + random.uniform(0, wait / 2)
            heapq.heappush(events, (t + wait, c, n + 1))
        else:
            served[sec] = served.get(sec, 0) + 1; done_at = max(done_at, t)
    return {"jitter": jitter, "all_done_s": done_at, "busiest_second": max(arrivals.values()), "attempts": attempts}

rows = [run(m) for m in ("none", "equal", "full")]
for r in rows: print(f"jitter {r['jitter']:<6} all served after {r['all_done_s']:6.1f} s   busiest second {r['busiest_second']:>4} arrivals   attempts {r['attempts']:,}")

if args.json:
    with open(args.json, "w") as f:
        json.dump({"tables": {"modes": rows}}, f)

3Run it in the folder you saved it to, with the defaults first. Then change a knob from the table below and run it again.
Terminal
```
uv run retry-storm-no-jitter-versus-full.py
uv run retry-storm-no-jitter-versus-full.py --help     # every knob
```

Knobs
Knob · default	What it changes
--clients400	Clients.
--capacity50	Requests served per second.
--down2	Seconds the server is down at the start.
--seed0	Random seed.

Our run

Three backoff policies, 400 clients

Three backoff policies, 400 clients
Jitter	All served after	Busiest second (arrivals)	Attempts
none	62.7 s	1.6K	3.8K
equal	20.5 s	1.66K	3K
full	14.8 s	1.98K	3.1K

Reference runApple M2 · 8 cores · 8 GB

Output · uv run retry-storm-no-jitter-versus-full.py

jitter none   all served after   62.7 s   busiest second 1600 arrivals   attempts 3,800
jitter equal  all served after   20.5 s   busiest second 1661 arrivals   attempts 3,002
jitter full   all served after   14.8 s   busiest second 1981 arrivals   attempts 3,103

macOS 26.6.2 · Python 3.13.3 · run 2026-10-20

What we found

Without jitter the last client was served after 62.7 s; with equal jitter after 20.5 s, and with full jitter after 14.8 s, as expected. Attempts did not follow the expectation: no jitter made 3.8K, full jitter 3.1K, and equal jitter the fewest, 3K. Full jitter draws some waits close to zero, so a client can retry several times quickly while the server is still down. Jitter did not lower the busiest second: it saw 1.6K arrivals without jitter and 1.98K with full jitter.

Read it

Jitter's value is in completion time: it turns waves that the server rejects into a stream it can serve, so the backlog clears in a fraction of the time. The attempt count depends on the jitter shape; full jitter wastes a few early retries in exchange for the fastest spread. Neither shape helps the first second, when every client fails at once; that is what a retry budget or a client-side circuit breaker is for.