Skip to content
About
Main content
Model

Brute-force vector search is memory-bound

Vector search is memory-bandwidth-bound, not compute-bound.

Vectors
vectors scanned
Dimensions
dimensions per vector (float32)

Recurring principle: Granularity mismatch

How it behaves

One input on the horizontal axis, the others held fixed.

Data table
Brute-force query time by corpus size (768 dims, 50 GB/s)
Vectors NTime per query
100K6.14 ms
115K7.09 ms
133K8.19 ms
154K9.46 ms
178K10.9 ms
205K12.6 ms
237K14.6 ms
274K16.8 ms
316K19.4 ms
365K22.4 ms
422K25.9 ms
487K29.9 ms
562K34.6 ms
649K39.9 ms
750K46.1 ms
866K53.2 ms
1M61.4 ms
1.15M70.9 ms
1.33M81.9 ms
1.54M94.6 ms
1.78M109 ms
2.05M126 ms
2.37M146 ms
2.74M168 ms
3.16M194 ms
3.65M224 ms
4.22M259 ms
4.87M299 ms
5.62M346 ms
6.49M399 ms
7.5M461 ms
8.66M532 ms
10M614 ms
11.5M709 ms
13.3M819 ms
15.4M946 ms
17.8M1.09 s
20.5M1.26 s
23.7M1.46 s
27.4M1.68 s
31.6M1.94 s
36.5M2.24 s
42.2M2.59 s
48.7M2.99 s
56.2M3.46 s
64.9M3.99 s
75M4.61 s
86.6M5.32 s
100M6.14 s
115M7.09 s
133M8.19 s
154M9.46 s
178M10.9 s
205M12.6 s
237M14.6 s
274M16.8 s
316M19.4 s
365M22.4 s
422M25.9 s
487M29.9 s
562M34.6 s
649M39.9 s
750M46.1 s
866M53.2 s
1B61.4 s
1.15B70.9 s
1.33B81.9 s
1.54B94.6 s
1.78B109 s
2.05B126 s
2.37B146 s
2.74B168 s
3.16B194 s
3.65B224 s
4.22B259 s
4.87B299 s
5.62B346 s
6.49B399 s
7.5B461 s
8.66B532 s
10B614 s

Worked examples, recomputed

Every number the mechanisms quote for this relationship is recomputed from it, so none of them disagrees with it.

Where it appears

Each mechanism states it in its own names. The general symbol each name stands for is shown with it.