Float summation is not associative
Measure your machineNeeds Python
Problem
Floating-point addition rounds after every step, so the result depends on the order of the additions: adding a small number to a large one can lose it entirely, and the same numbers added in a different order lose different bits. A distributed SUM adds in an order set by partitioning and scheduling, so its result changes when the cluster is resized. An exactly rounded sum (math.fsum, a relative of Kahan summation) gives one stable value.
Sum one million doubles of very different magnitudes in five random orders with a plain left-to-right loop (Python 3.12's built-in sum() already compensates, so it would hide the effect), and compare with math.fsum.
The question. How many different sums do five orders give, and how far are they from the exactly rounded sum?
Predict
Several different sums for the same data (they differ in the low digits), and math.fsum gives one stable value.
Run it
1Install uv once. It fetches Python and the lab's packages for you.
Terminal curl -LsSf https://astral.sh/uv/install.sh | sh2Get the script. It is the whole lab: read it, then download it.
View the codeHide the code
float-summation-is-not-associative.py # /// script # requires-python = ">=3.10" # /// # Copyright (c) 2026 Aayush Sharma · SPDX-License-Identifier: MIT # https://thedataengineering.com/lab-code/LICENSE.txt """Float summation is not associative. One million doubles of very different magnitudes, summed in five random orders with a plain left-to-right loop (Python 3.12's built-in sum() compensates, so it would hide the effect), then with math.fsum, which is exactly rounded. """ import argparse, json, math, random p = argparse.ArgumentParser(description=__doc__.splitlines()[0]) p.add_argument("--n", type=int, default=1_000_000, help="Numbers.") p.add_argument("--orders", type=int, default=5, help="Random orders to sum in.") p.add_argument("--seed", type=int, default=0, help="Random seed.") p.add_argument("--json", help="Also write the results to this JSON file.") args = p.parse_args() random.seed(args.seed) xs = [random.uniform(-1, 1) * 10 ** random.randint(0, 12) for _ in range(args.n)] def plain(v): s = 0.0 for x in v: s += x # what a simple engine or a reducer does return s rows = [] for i in range(args.orders): random.shuffle(xs); rows.append({"order": i + 1, "plain_sum": plain(xs), "fsum": math.fsum(xs)}) exact = math.fsum(xs) for r in rows: r["plain_minus_exact"] = r["plain_sum"] - exact summary = {"distinct_plain": len({r["plain_sum"] for r in rows}), "distinct_fsum": len({r["fsum"] for r in rows}), "exact": exact, "max_abs_error": max(abs(r["plain_minus_exact"]) for r in rows)} for r in rows: print(f"order {r['order']}: plain loop {r['plain_sum']!r:<24} fsum {r['fsum']!r}") print(f"{summary['distinct_plain']} different plain sums, {summary['distinct_fsum']} distinct fsum value(s); largest error {summary['max_abs_error']:.4g}") if args.json: with open(args.json, "w") as f: json.dump({"summary": summary, "tables": {"orders": rows}}, f)3Run it in the folder you saved it to, with the defaults first. Then change a knob from the table below and run it again.
Terminal uv run float-summation-is-not-associative.py uv run float-summation-is-not-associative.py --help # every knob
| Knob · default | What it changes |
|---|---|
| --n1,000,000 | Numbers. |
| --orders5 | Random orders to sum in. |
| --seed0 | Random seed. |
Our run
The same numbers in five orders
| Order | Plain loop | math.fsum | Plain − exact |
|---|---|---|---|
| 1 | 83233593475878.28 | 83233593475876.86 | 1.42 |
| 2 | 83233593475879.89 | 83233593475876.86 | 3.03 |
| 3 | 83233593475876.83 | 83233593475876.86 | -0.031 |
| 4 | 83233593475880.97 | 83233593475876.86 | 4.11 |
| 5 | 83233593475878.98 | 83233593475876.86 | 2.13 |
order 1: plain loop 83233593475878.28 fsum 83233593475876.86
order 2: plain loop 83233593475879.89 fsum 83233593475876.86
order 3: plain loop 83233593475876.83 fsum 83233593475876.86
order 4: plain loop 83233593475880.97 fsum 83233593475876.86
order 5: plain loop 83233593475878.98 fsum 83233593475876.86
5 different plain sums, 1 distinct fsum value(s); largest error 4.109What we found
Five orders gave 5 different plain sums, off by up to 4.11 from the exactly rounded value; math.fsum gave 1 value in every order.
Read it
This is why a distributed SUM of doubles changes when the cluster is resized, and why money belongs in decimal or fixed-point types. If floats are unavoidable, use compensated or exactly rounded summation, or accept and document a tolerance in every reconciliation.