Decide whether to specialize
A new workload arrives: full-text search, similarity search over embeddings, friend-of-friend queries, location lookups, high-volume metrics. A specialized engine promises to be far faster. But the real cost of a specialized system is rarely the engine: it is a second copy of the data, kept in sync, forever. Do you need one?
You will end with general engine, extension or specialized engine, with the pipeline cost and the unverifiable parts stated.
Decide
Can the query's condition be expressed as a range or an equality on some key?
'Orders in the last hour' or 'user_id = 42' can; 'documents mentioning bicycle', 'the 10 most similar vectors', 'within 2 km' and 'three hops from me' usually cannot directly.
Why
B-trees, partition pruning and hash joins all rest on one assumption: the predicate reduces to a point or range in a sorted order, or an exact hash match. When it does, a general database already does this well.
Does an extension to your general engine get within the performance budget?
Full-text, vector and geospatial extensions for PostgreSQL, for example. Measure against your actual budget, not against the specialized engine's benchmark.
Why
An extension keeps one copy of the data, one write path and one source of truth. The line between "extension is enough" and "needs its own system" is economic and moves every year as general engines improve.
Where you can land
Stay in the general engine
Use it when the condition reduces to a range or an equality.
Index the key; nothing else needed.
Use an extension
Use it when the general engine with an extension meets the budget.
What it costs: less speed than a dedicated engine, and a ceiling you may reach later. Re-check the budget as volumes grow.
Adopt a specialized engine
Use it when nothing general meets the budget.
Before you build it
Name the shape of the problem
Which way does the system escape the limits of ordinary indexes? Map the data to a space where a cheap test prunes, then check survivors exactly (text, geospatial, vectors); change the execution model (graph traversal); store the data in a specialized compact form (time series); or, where no answer can be verified, rely on reproducibility (ML features).
Name the binding resource and its knob, and who owns it
| Family | Binding resource | The knob |
|---|---|---|
| Text | postings scanned | early-termination threshold |
| Vector | memory bandwidth | recall settings, bits per vector |
| Graph | coordination per hop | traversal depth |
| Geospatial | exact-check work | cell resolution |
| Time series | number of series, bytes per point | rollup resolution, retention |
| ML features | freshness | snapshot depth |
Price the pipeline, not only the engine
A second write path, a freshness metric, a reconciliation job against the source of truth, a deletion path (so erased data disappears from the new system too), and a measured rebuild time, which becomes your recovery time.
Say what you cannot verify, and who finds out first
Approximate search returns plausible results when it misses; relevance has no ground truth. Write down what nobody can check, and who will notice when it is wrong.