The memory-bound arithmetic of brute-force vector search

Measure your machineNeeds Python with NumPy

Problem

A brute-force vector search reads every vector once and does one multiply-add per dimension: two FLOPs per four bytes in float32, far below any CPU's ridge point, so it should be memory-bound and its time should scale with the bytes read. Storing vectors as float16 or int8 halves or quarters the bytes, which should halve or quarter the time, if the arithmetic on those types is as fast as on float32.

With NumPy, create random float32 vectors of dimension 768 (400,000 of them, 1.2 GB, scaled down from a million for an 8 GB laptop). Time one brute-force query: scores = vectors @ q, then np.argpartition for the top 10. Compute the effective bandwidth as bytes read divided by seconds. Repeat after storing the vectors as float16 and as int8. Scale the float32 result to 10 million vectors.

The question. Is the query memory-bound, and does halving or quartering the bytes speed it up accordingly?

Predict

Storing the vectors as float16, half the bytes of float32: how does the brute-force query time change in NumPy?

About halfAbout the sameSomewhat slowerMany times slower

Run it

1Install uv once. It fetches Python and the lab's packages for you.
Terminal
```
curl -LsSf https://astral.sh/uv/install.sh | sh
```

2Get the script. It is the whole lab: read it, then download it.

the-memory-bound-arithmetic-of-brute.py42 linesDownload

View the code

the-memory-bound-arithmetic-of-brute.py

# /// script
# requires-python = ">=3.10"
# dependencies = ["numpy"]
# ///
# Copyright (c) 2026 Aayush Sharma · SPDX-License-Identifier: MIT
# https://thedataengineering.com/lab-code/LICENSE.txt
"""The memory-bound arithmetic of brute-force search.

400,000 random 768-dimensional vectors (1.2 GB as float32). One brute-force
query (scores = vectors @ q, then argpartition for the top 10), timed as
float32, float16 and int8; effective bandwidth = bytes read ÷ seconds; and the
float32 time scaled to 10 million vectors.
"""
import argparse, json, time
import numpy as np

p = argparse.ArgumentParser(description=__doc__.splitlines()[0])
p.add_argument("--n", type=int, default=400_000, help="Vectors.")
p.add_argument("--dim", type=int, default=768, help="Dimension.")
p.add_argument("--repeats", type=int, default=5, help="Queries per type (best kept).")
p.add_argument("--json", help="Also write the results to this JSON file.")
args = p.parse_args()

rng = np.random.default_rng(0)
v32 = rng.standard_normal((args.n, args.dim), dtype=np.float32); q32 = rng.standard_normal(args.dim, dtype=np.float32)
rows = []
for name, v, q in (("float32", v32, q32), ("float16", v32.astype(np.float16), q32.astype(np.float16)),
                   ("int8", np.clip(v32 * 32, -127, 127).astype(np.int8), np.clip(q32 * 32, -127, 127).astype(np.int8))):
    ts = []
    for _ in range(args.repeats):
        t = time.perf_counter(); s = v @ q; top = np.argpartition(s, -10)[-10:]; ts.append(time.perf_counter() - t)
    el = min(ts); rows.append({"type": name, "gb": v.nbytes / 1e9, "ms": el * 1000, "gb_s": v.nbytes / el / 1e9})
    if name != "float32": del v
for r in rows: r["vs_float32"] = r["ms"] / rows[0]["ms"]
summary = {"ms_10m_float32": rows[0]["ms"] * 10_000_000 / args.n}
for r in rows: print(f"{r['type']:<8} {r['gb']:5.2f} GB  one query {r['ms']:8.1f} ms  ({r['gb_s']:5.1f} GB/s, {r['vs_float32']:.2f}x float32's time)")
print(f"float32 scaled to 10 million vectors: {summary['ms_10m_float32'] / 1000:.1f} s per query")

if args.json:
    with open(args.json, "w") as f:
        json.dump({"summary": summary, "tables": {"types": rows}}, f)

3Run it in the folder you saved it to, with the defaults first. Then change a knob from the table below and run it again.
Terminal
```
uv run the-memory-bound-arithmetic-of-brute.py
uv run the-memory-bound-arithmetic-of-brute.py --help     # every knob
```

Knobs
Knob · default	What it changes
--n400,000	Vectors.
--dim768	Dimension.
--repeats5	Queries per type (best kept).

Our run

One brute-force query

One brute-force query
Type	GB	Time (ms)	GB/s	vs float32
float32	1.23	24.6	49.9	1×
float16	0.614	493	1.25	20×
int8	0.307	113	2.71	4.61×

Reference runApple M2 · 8 cores · 8 GB

Output · uv run the-memory-bound-arithmetic-of-brute.py

float32   1.23 GB  one query     24.6 ms  ( 49.9 GB/s, 1.00x float32's time)
float16   0.61 GB  one query    492.6 ms  (  1.2 GB/s, 20.02x float32's time)
int8      0.31 GB  one query    113.3 ms  (  2.7 GB/s, 4.61x float32's time)
float32 scaled to 10 million vectors: 0.6 s per query

macOS 26.6.2 · Python 3.13.3 · numpy 2.5.4 · run 2026-10-20

What we found

The float32 query read 1.23 GB in 24.6 ms, 49.9 GB/s: memory-bound, near the 50 GB/s assumed. Smaller types were slower, not faster: float16 took 493 ms and int8 113 ms, because NumPy has fast BLAS kernels only for float32 and float64, and multiplies the others in generic loops, so they became compute-bound. Scaled to 10 million vectors, a float32 query takes 615 ms.

Read it

The memory-bound arithmetic is right; whether a smaller type is faster depends on whether the hardware and library have fast kernels for it. Vector databases use SIMD kernels for int8 and float16 precisely because generic matrix libraries do not, and on those kernels the byte arithmetic holds.