The power of two choices
Simulate a modelNeeds Python
Problem
Placing tasks on random machines is cheap and spreads load evenly on average, but the busiest machine ends up with about log n / log log n tasks. Asking two random machines and taking the less loaded one drops that to about log log n: an exponential improvement from a single extra probe. A third probe adds very little.
Throw n = 100,000 tasks at n = 100,000 machines. For each task, choose (a) one random machine, (b) the less loaded of two random machines, (c) the least loaded of three. Record the maximum load, over five seeds.
The question. What is the maximum load with one, two and three choices?
Predict
One choice gives a maximum load of about 7–9. Two choices give about 3–4. Three choices give about 3. Most of the gain comes from the second probe.
Run it
1Install uv once. It fetches Python and the lab's packages for you.
Terminal curl -LsSf https://astral.sh/uv/install.sh | sh2Get the script. It is the whole lab: read it, then download it.
View the codeHide the code
the-power-of-two-choices.py # /// script # requires-python = ">=3.10" # /// # Copyright (c) 2026 Aayush Sharma · SPDX-License-Identifier: MIT # https://thedataengineering.com/lab-code/LICENSE.txt """The power of two choices. 100,000 tasks thrown at 100,000 machines: each task goes to one random machine, to the less loaded of two random machines, or to the least loaded of three. The maximum load, over several seeds. """ import argparse, json, random p = argparse.ArgumentParser(description=__doc__.splitlines()[0]) p.add_argument("--n", type=int, default=100_000, help="Tasks and machines.") p.add_argument("--seeds", type=int, default=5, help="Runs per policy.") p.add_argument("--json", help="Also write the results to this JSON file.") args = p.parse_args() def run(choices): load = [0] * args.n for _ in range(args.n): picks = [random.randrange(args.n) for _ in range(choices)] m = min(picks, key=lambda i: load[i]); load[m] += 1 return max(load) rows = [] for c in (1, 2, 3): maxes = [] for s in range(args.seeds): random.seed(s); maxes.append(run(c)) rows.append({"choices": c, "max_load_min": min(maxes), "max_load_max": max(maxes), "max_loads": ", ".join(map(str, maxes))}) for r in rows: print(f"{r['choices']} choice{'s' if r['choices'] > 1 else ' '}: maximum load {r['max_loads']}") if args.json: with open(args.json, "w") as f: json.dump({"tables": {"policies": rows}}, f)3Run it in the folder you saved it to, with the defaults first. Then change a knob from the table below and run it again.
Terminal uv run the-power-of-two-choices.py uv run the-power-of-two-choices.py --help # every knob
| Knob · default | What it changes |
|---|---|
| --n100,000 | Tasks and machines. |
| --seeds5 | Runs per policy. |
Our run
Maximum load over 5 seeds
| Choices | Maximum load per seed |
|---|---|
| 1 | 8, 7, 8, 7, 8 |
| 2 | 4, 4, 3, 3, 4 |
| 3 | 3, 3, 3, 3, 3 |
1 choice : maximum load 8, 7, 8, 7, 8
2 choices: maximum load 4, 4, 3, 3, 4
3 choices: maximum load 3, 3, 3, 3, 3What we found
Exactly as expected: one choice gave a maximum of 8, 7, 8, 7, 8, two choices 4, 4, 3, 3, 4, three choices 3, 3, 3, 3, 3.
Read it
The second probe is almost free and removes most of the imbalance; that is why load balancers, hash tables (cuckoo hashing) and schedulers sample two candidates rather than one or all. The catch in practice is stale load information: if every balancer sees the same stale loads, they all pick the same "least loaded" machine.