Nothing can be eliminated by inspection
Why is brute-force similarity search memory-bound, and why does compression buy speed first?
Text search worked because term membership is sparse and discriminating. A rare term names a thousand documents and silently eliminates the other ten million for free, because those documents are simply not in the postings list. The inverted index is a pruning device that costs nothing to consult.
Vectors have no such property. Every dimension of every vector is populated, and every vector has a defined similarity to every query. Nothing is eliminated by inspection. There is no set to intersect, no value to compare, no absence to exploit. The candidate set is the corpus, always. The only way to know that a vector is not among the top ten is to compute its distance, and then you have already paid the cost you were trying to avoid.
So the governing question inverts. In text it was: how do I avoid scoring the candidates I found? Here it is: how do I avoid finding them at all, when membership is a continuous quantity rather than a fact?
The constraint that stops mattering is exactness. The one that appears is geometric: pruning must come from spatial structure, and high-dimensional space has almost none to give.
This approximate task is called ANN (approximate nearest neighbour) search. Its quality is measured by recall: of the true top-k neighbours (found by exact brute-force), the fraction your index returned. A recall of 0.95 means that on average 95 of the true 100 nearest neighbours are returned.
The arithmetic that forces everything
One distance between two vectors of d dimensions costs d multiply-adds. A brute-force query over N vectors costs N × d. But the arithmetic is not the binding constraint. The memory traffic is.
Note the arithmetic intensity: two floating-point operations (a multiply and an add) are done per four bytes read, about 0.5 FLOP per byte. That is far below the ridge point of any modern machine. Vector search is memory-bandwidth-bound, not compute-bound.
This reorders the folklore. Quantization (storing vectors in fewer bits) is commonly explained as a way to fit the index in RAM. Capacity is real but secondary. Compressing vectors 32x multiplies the scan rate by roughly 32x as well, because you read 32 times fewer bytes for the same number of candidates.
One line: similarity search is memory-bound at half a flop per byte, so bytes per vector is the query cost, and compression buys speed before it buys space.