Skip to content
About
Main content

The multiplexing dividend and its collapse under correlation

Simulate a modelNeeds Python with NumPy

Problem

Shared capacity is cheaper than dedicated capacity because tenants rarely peak together: the peak of the sum is less than the sum of the peaks. The size of that dividend depends on correlation. Independent demands mostly cancel; demands driven by a common factor (the same morning, the same marketing campaign) peak together, and the dividend disappears.

Simulate 16 tenants whose demand is 10 + 5 * (sqrt(rho) * z + sqrt(1 - rho) * e_i), where z is one shared random factor and e_i is independent noise. Compare the 99th percentile of the sum of demands with the sum of the 99th percentiles, for rho = 0, 0.3, 0.6 and 1.

The question. How much capacity does sharing save at each correlation?


Predict

At ρ = 0.3, what share of the separate tenants' capacity does one shared pool need?

Run it

  1. 1Install uv once. It fetches Python and the lab's packages for you.

    Terminal
    curl -LsSf https://astral.sh/uv/install.sh | sh
  2. 2Get the script. It is the whole lab: read it, then download it.

    the-multiplexing-dividend-and-its-collapse.py35 linesDownload
    View the code
    the-multiplexing-dividend-and-its-collapse.py
    # /// script
    # requires-python = ">=3.10"
    # dependencies = ["numpy"]
    # ///
    # Copyright (c) 2026 Aayush Sharma · SPDX-License-Identifier: MIT
    # https://thedataengineering.com/lab-code/LICENSE.txt
    """The multiplexing dividend, and its collapse.
    
    16 tenants whose demand is 10 + 5·(√ρ·z + √(1−ρ)·eᵢ), with z a shared random
    factor and eᵢ independent noise. The 99th percentile of the summed demand
    against the sum of the 99th percentiles, for ρ from 0 to 1.
    """
    import argparse, json
    import numpy as np
    
    p = argparse.ArgumentParser(description=__doc__.splitlines()[0])
    p.add_argument("--tenants", type=int, default=16, help="Tenants.")
    p.add_argument("--samples", type=int, default=200_000, help="Time samples.")
    p.add_argument("--rhos", default="0,0.3,0.6,1.0", help="Correlations.")
    p.add_argument("--seed", type=int, default=0, help="Random seed.")
    p.add_argument("--json", help="Also write the results to this JSON file.")
    args = p.parse_args()
    
    rng = np.random.default_rng(args.seed); rows = []
    for rho in map(float, args.rhos.split(",")):
        z = rng.standard_normal((args.samples, 1)); e = rng.standard_normal((args.samples, args.tenants))
        d = 10 + 5 * (np.sqrt(rho) * z + np.sqrt(1 - rho) * e)
        shared = float(np.percentile(d.sum(axis=1), 99)); separate = float(sum(np.percentile(d[:, i], 99) for i in range(args.tenants)))
        rows.append({"rho": rho, "shared": shared, "separate": separate, "ratio": shared / separate})
    for r in rows: print(f"rho={r['rho']:.1f}  shared={r['shared']:6.1f}  separate={r['separate']:6.1f}  ratio={r['ratio']:.2f}")
    
    if args.json:
        with open(args.json, "w") as f:
            json.dump({"tables": {"rhos": rows}}, f)
    
  3. 3Run it in the folder you saved it to, with the defaults first. Then change a knob from the table below and run it again.

    Terminal
    uv run the-multiplexing-dividend-and-its-collapse.py
    uv run the-multiplexing-dividend-and-its-collapse.py --help     # every knob
Knobs
Knob · defaultWhat it changes
--tenants16 Tenants.
--samples200,000 Time samples.
--rhos0, 0.3, 0.6, 1.0 Correlations.
--seed0 Random seed.

Our run

Shared p99 ÷ sum of separate p99sApple M2 · 8 cores · 8 GB
Data table
Shared p99 ÷ sum of separate p99s
Correlation ρRatio
00.596
0.30.778
0.60.888
11
macOS 26.6.2 · Python 3.13.3 · numpy 2.5.4 · run 2026-10-20
Output · uv run the-multiplexing-dividend-and-its-collapse.py
rho=0.0  shared= 206.5  separate= 346.3  ratio=0.60
rho=0.3  shared= 269.4  separate= 346.0  ratio=0.78
rho=0.6  shared= 307.4  separate= 346.1  ratio=0.89
rho=1.0  shared= 346.7  separate= 346.7  ratio=1.00

What we found

The ratio was 0.596 with independent demand, 0.778 at ρ = 0.3, 0.888 at 0.6 and 1 at full correlation, as expected.


Read it

Even modest correlation takes away much of the dividend: at ρ = 0.3 a large part of the saving is gone. Capacity plans that assume tenants peak independently are optimistic exactly when it matters, on the days a common cause drives everyone at once.