Segments and near-real-time refresh
How does a text index stay fresh, and what does each refresh cost queries?
Inserting one document into a globally sorted inverted index means touching every postings list that document's terms appear in: hundreds of random, small, in-place writes per document, each of which must re-encode a delta-compressed run. That is the write pattern that all modern storage design exists to avoid.
So nobody does it. The design instead is:
- Buffer incoming documents in memory.
- Periodically flush a small, immutable, self-contained index segment.
- Search all segments and merge their results.
- Delete by tombstone.
- Merge segments in the background.
This is the LSM design, unmodified in structure, arriving in yet another domain. The reliable generalization: any structure that is expensive to update in place and cheap to read in sorted order converges on this shape. The interesting question is never whether, but what the merge debt looks like.
The merge debt of a text index
A tombstoned document still occupies space in every postings list it appears in. It still has to be read during query evaluation. And it still has to be filtered out after it has influenced skipping decisions and before it can be counted. So deleted data costs query latency, not merely space, until compaction reclaims it.
Refresh interval and the per-segment fixed cost
Visibility latency equals the flush interval: a document is unsearchable until it lands in a segment. A shorter interval gives fresher results and more segments.
But query cost carries a per-segment fixed term. Every segment has its own term dictionary, so every query term costs one dictionary lookup per segment before any postings are read:
This is the rule that a fixed cost per operation sets a natural batch size, with one twist worth naming: the fixed cost is paid by readers, while the batch size is chosen by the write path. The writer picks the refresh interval; the reader absorbs the consequence. The cost has been relocated onto another party. It is the reason merge policy is tuned against query latency rather than write throughput.
The conventional one-second refresh is a default, not a physical constant. It is a reasonable compromise for interactive search. It is an actively bad one for a log-ingestion workload where nobody reads the last second, and also for a catalogue where staleness is visible to a customer.
Global statistics versus local segments
Finally, a correctness wrinkle that segmentation creates and most systems knowingly accept. IDF is a global statistic, a term's rarity across the whole corpus. But segments and shards hold local document counts. Scored independently, the same document can receive different scores depending on which shard it landed in, and two documents can be ordered wrongly relative to each other. Exact global scoring requires an extra round trip: a first round collects term statistics from all shards, a second queries with them (the same two-round scatter-gather gives exact global top-k). Almost no one pays for it at query time, because with well-mixed sharding the local estimate is close, and the errors are invisible against a relevance function nobody can verify anyway.
That last clause is the whole article in miniature:
One line: the index is immutable and merged, freshness is a flush interval, and the per-segment fixed cost means the writer's batch choice is billed to every reader.