Skip to content
About
Main content

The power of two choices

Simulate a modelNeeds Python

Problem

Placing tasks on random machines is cheap and spreads load evenly on average, but the busiest machine ends up with about log n / log log n tasks. Asking two random machines and taking the less loaded one drops that to about log log n: an exponential improvement from a single extra probe. A third probe adds very little.

Throw n = 100,000 tasks at n = 100,000 machines. For each task, choose (a) one random machine, (b) the less loaded of two random machines, (c) the least loaded of three. Record the maximum load, over five seeds.

The question. What is the maximum load with one, two and three choices?


Predict

Going from two random choices to three: how much does the maximum load drop?

Run it

  1. 1Install uv once. It fetches Python and the lab's packages for you.

    Terminal
    curl -LsSf https://astral.sh/uv/install.sh | sh
  2. 2Get the script. It is the whole lab: read it, then download it.

    the-power-of-two-choices.py36 linesDownload
    View the code
    the-power-of-two-choices.py
    # /// script
    # requires-python = ">=3.10"
    # ///
    # Copyright (c) 2026 Aayush Sharma · SPDX-License-Identifier: MIT
    # https://thedataengineering.com/lab-code/LICENSE.txt
    """The power of two choices.
    
    100,000 tasks thrown at 100,000 machines: each task goes to one random
    machine, to the less loaded of two random machines, or to the least loaded of
    three. The maximum load, over several seeds.
    """
    import argparse, json, random
    
    p = argparse.ArgumentParser(description=__doc__.splitlines()[0])
    p.add_argument("--n", type=int, default=100_000, help="Tasks and machines.")
    p.add_argument("--seeds", type=int, default=5, help="Runs per policy.")
    p.add_argument("--json", help="Also write the results to this JSON file.")
    args = p.parse_args()
    
    def run(choices):
        load = [0] * args.n
        for _ in range(args.n):
            picks = [random.randrange(args.n) for _ in range(choices)]
            m = min(picks, key=lambda i: load[i]); load[m] += 1
        return max(load)
    rows = []
    for c in (1, 2, 3):
        maxes = []
        for s in range(args.seeds): random.seed(s); maxes.append(run(c))
        rows.append({"choices": c, "max_load_min": min(maxes), "max_load_max": max(maxes), "max_loads": ", ".join(map(str, maxes))})
    for r in rows: print(f"{r['choices']} choice{'s' if r['choices'] > 1 else ' '}: maximum load {r['max_loads']}")
    
    if args.json:
        with open(args.json, "w") as f:
            json.dump({"tables": {"policies": rows}}, f)
    
  3. 3Run it in the folder you saved it to, with the defaults first. Then change a knob from the table below and run it again.

    Terminal
    uv run the-power-of-two-choices.py
    uv run the-power-of-two-choices.py --help     # every knob
Knobs
Knob · defaultWhat it changes
--n100,000 Tasks and machines.
--seeds5 Runs per policy.

Our run

Maximum load over 5 seeds

Maximum load over 5 seeds
ChoicesMaximum load per seed
18, 7, 8, 7, 8
24, 4, 3, 3, 4
33, 3, 3, 3, 3
Reference runApple M2 · 8 cores · 8 GB
Output · uv run the-power-of-two-choices.py
1 choice : maximum load 8, 7, 8, 7, 8
2 choices: maximum load 4, 4, 3, 3, 4
3 choices: maximum load 3, 3, 3, 3, 3
macOS 26.6.2 · Python 3.13.3 · run 2026-10-20

What we found

Exactly as expected: one choice gave a maximum of 8, 7, 8, 7, 8, two choices 4, 4, 3, 3, 4, three choices 3, 3, 3, 3, 3.


Read it

The second probe is almost free and removes most of the imbalance; that is why load balancers, hash tables (cuckoo hashing) and schedulers sample two candidates rather than one or all. The catch in practice is stale load information: if every balancer sees the same stale loads, they all pick the same "least loaded" machine.