Glossary

133 terms, in the sense this mechanism uses them. The platform glossary has every term across all mechanisms.

A/B test: Comparing two versions under the same conditions to see if the difference is real.You compare two versions, A and B, under the same conditions and ask whether the difference is real. An A/A test compares the same version against itself, which shows how large the difference can be from noise alone.
Adjacency list: Storage that keeps each vertex's neighbours together in one contiguous run.
Alignment (of series): Matching the timestamps of independently sampled series, usually by resampling or an as-of join.
Analyzer: The index-time and query-time pipeline that turns text into tokens (splitting, lowercasing, stemming, stop words).
ANN (approximate nearest neighbor): Search that returns neighbours that are probably, not certainly, the closest.
Arithmetic intensity: FLOPs done per byte moved from memory; the x-axis of the roofline.
As-of join: A join that, for each left row at time t, picks the right row with the greatest time at or before t.A join that, for each event at time t, picks the most recent value of another table that was valid at or before t.
Availability time: The moment a feature value could really be used by a consumer (after its pipeline ran), as opposed to the time it describes.
Batch inference: Applying a model to many rows in a pipeline; a third materialization of features that needs the same point-in-time care as training.
Bidirectional search: Searching from both endpoints and meeting in the middle, costing about 2*d^(k/2) instead of d^k.
Bitemporal storage: Storing both when a fact was true (valid time) and when the system learned it (transaction time).
Bitmap encodings: Encodings that represent a set as a string of bits, where bit i says whether item i is in the set; often combined with dictionary and delta encoding in columnar indexes.Dictionary encoding replaces repeated values with small integer codes. Delta encoding stores the difference from the previous value instead of the value itself (the list 1000, 1002, 1005 becomes 1000, +2, +3). A bitmap is a string of bits where bit i says whether item i is in a set.
Block-max / WAND: Algorithms that skip blocks of postings whose maximum possible score cannot beat the current tenth-best score.
BM25: A standard text relevance score that sums per-term contributions based on term frequency, document length and rarity.
Bounding box: The smallest axis-aligned rectangle that encloses a geometry, used as a cheap one-sided filter.
Branching factor: Average number of neighbours per vertex, which sets how fast a traversal frontier grows.
Bulk synchronous parallel (BSP): Computation in rounds separated by a global barrier, where messages sent in a round arrive together in the next.
Bε-tree (fractal tree): A B-tree whose nodes reserve space for buffered updates; a continuum between B-tree and LSM.
Cardinality: The number of distinct values a label can take. The series count is the product over all labels.2 other meanings →The number of distinct things in a set. "20 million files" is a statement about cardinality. Metadata size follows cardinality (how many objects), not bytes.
Cell id: A string or integer naming a grid cell, such that child cells share the parent's prefix.
Columnar layout: Storing each column's values together.Store each column's values together instead of storing each row together. A query that needs only one column reads only that column.
Combiner: A step that aggregates messages per destination before they are sent, as partial aggregation does.
Compaction: Rewriting many small files into fewer larger ones.An LSM-tree (Log-Structured Merge tree) is a storage structure that never changes files in place. It collects writes in memory, writes them as sorted, immutable files, and later compacts (merges and rewrites) the files in the background to remove old versions and deleted entries. A tombstone is a marker that says "this key was deleted"; compaction removes the key and the marker. Compaction costs disk and CPU. Write amplification is the ratio of bytes the storage actually writes to bytes the application asked to write. A Bloom filter is a small structure that answers "definitely not in this file" or "maybe in this file", so reads can skip most files.
Connected component: A maximal set of vertices that can reach each other.
Coordinate reference system (CRS): The definition of what a coordinate pair means, such as lat/long on an ellipsoid or metres in a projection.
CSR (compressed sparse row): Graph layout of one neighbour-id array plus one offset array indexed by vertex id.
Data-source skew: Offline and online features come from different stores populated by different pipelines.
Decomposability: A score that is a sum of independent per-term parts, each with a computable maximum.
Degree: The number of edges attached to a vertex.
Delta: Storing the difference from the previous value instead of the value itself: 1000, 1002, 1005 becomes 1000, +2, +3. Small differences fit in few bits.1 other meaning →Dictionary encoding replaces repeated values with small integer codes. Delta encoding stores the difference from the previous value instead of the value itself (the list 1000, 1002, 1005 becomes 1000, +2, +3). A bitmap is a string of bits where bit i says whether item i is in a set.
Delta encoding: Storing differences between consecutive values instead of the values.
Delta-of-delta: Encoding the change in the change of a value; for regular timestamps this is mostly zero.
Dictionary: Dictionary encoding: replacing repeated values with small integer codes and storing each distinct value once. A column with few distinct values becomes very small.Dictionary encoding replaces repeated values with small integer codes. Delta encoding stores the difference from the previous value instead of the value itself (the list 1000, 1002, 1005 becomes 1000, +2, +3). A bitmap is a string of bits where bit i says whether item i is in a set.
Dictionary encoding: Replacing each value by a small integer index into a table of distinct values.
Distribution skew (drift): The world changes so that live data no longer resembles training data; only monitoring detects it.
Downsampling: Reducing a series to fewer points, for example by averaging into larger time buckets.
Early binding: Checking the format when data is written, so errors appear in the producer.Binding means fixing the structure and meaning of data. Early binding: decide at load time ("this column is the customer's region"). Late binding: store raw facts and decide at read time. Early binding makes reads simple and changes expensive. Late binding makes changes cheap and reads complex.
Edge-cut partitioning: Assigning each vertex to one machine so that edges cross machine boundaries.
efSearch: The HNSW setting that controls how many candidates are explored per query.
Eight earlier rules: The analysis here refers to eight general rules about storage and query systems. Here they are in one place, each with a name we will use:
Embedding: A list of numbers (a vector) that represents the meaning of an item.
Event time: When something happened in the world, stamped near the source.Every event carries a timestamp of when it happened (event time), which differs from when the job sees it (processing time). Events arrive late and out of order: a phone with no signal sends its events 5 minutes after they happened. A watermark is a marker flowing through the stream that says "no event with a timestamp earlier than T should arrive any more." It is the job's best guess of how complete the input is. When the watermark passes the end of a time period, the job may treat that period as complete. Watermarks are a guess, and late events can still violate them. The rules of when it is correct to emit a result (event time versus processing time, how late is too late) are a topic of their own. In this mechanism we need only the idea above. We study the machinery that holds data until the watermark allows an answer.
Exchangeable rows: Rows whose order and grouping carry no information; the assumption behind a random split.
Facet: A count of matches grouped by a field value, shown beside search results.
False negative: A check that stays silent although the data is bad.A check can be wrong in two ways. A false positive (a false alarm) fires when the data is fine. A false negative (a miss) stays silent when the data is bad. Precision is: of the alarms that fired, what fraction were real defects. Recall is: of the real defects, what fraction did an alarm catch. You cannot push both false positives and false negatives to zero at the same time. You move a threshold and buy one by paying with the other.
False positive: A check that fires although the data is fine.A check can be wrong in two ways. A false positive (a false alarm) fires when the data is fine. A false negative (a miss) stays silent when the data is bad. Precision is: of the alarms that fired, what fraction were real defects. Recall is: of the real defects, what fraction did an alarm catch. You cannot push both false positives and false negatives to zero at the same time. You move a threshold and buy one by paying with the other.
Feature: A computed input to a model, such as average order value over 30 days.
Feature store: A system that defines a feature once and materializes it to an offline store for training and an online store for serving.
Fill ratio: Geometry area divided by bounding-box area. A low ratio means many false positives.
Filter-and-refine: A cheap filter with no false negatives, then an exact check on the survivors.A two-step pattern. First a cheap filter removes most candidates, with no false negatives. Then an expensive exact check (the refine step) runs only on the survivors. This mechanism will show it again and again.
Filtered ANN: Nearest-neighbour search restricted to items matching a predicate.
Fixpoint iteration: Repeating a step until no new rows appear.
Forward index: A structure that maps each document to its values, used for facets and sorting.
Frontier: The set of vertices reached in the current hop of a traversal.
Gap filling: Choosing a value for missing points, for example last value carried forward, linear interpolation or zero.
Geohash / S2 / H3: Discrete global grids that map a location to a hierarchical cell id.
Gorilla encoding: Time-series compression using delta-of-delta for timestamps and XOR for float values.
Group split: A train/test split that keeps all rows of one group (such as one courier) on the same side.
Hash index: An index that answers equality only, by hashing the key.
Hash-based split: A split decided by a hash of a stable key, so it is the same on every rerun.
HNSW: A layered neighbour graph that finds nearest neighbours by greedy navigation.
HyperLogLog: A distinct-count sketch that keeps the longest leading-zero run per bucket; error 1.04/√m.
Index-free adjacency: Storing neighbour lists directly with the vertex so a hop skips an index descent.
Intrinsic dimension: The effective number of dimensions the data really varies in, often much lower than the stored dimension.
Inverted index: A map from each term to the list of documents that contain it.
IVF (inverted file): An index that partitions vectors into cells by centroid and probes only a few cells.
Label leakage: Training with information that would not have been known at prediction time; it inflates the offline metric.
Late binding: Checking the format only when data is read, so errors appear in the consumer, far from the cause.Binding means fixing the structure and meaning of data. Early binding: decide at load time ("this column is the customer's region"). Late binding: store raw facts and decide at read time. Early binding makes reads simple and changes expensive. Late binding makes changes cheap and reads complex.
Late-interaction (multi-vector) retrieval: Retrieval where each document is represented by many vectors.
Little's Law: L = λW: average items in a system = arrival rate × average time in the system.Throughput is work finished per second. Latency is how long one operation takes. Little's Law links them to the number of operations in progress: L = λ × W, where λ is the arrival rate (per second), W is the average time one operation spends in the system, and L is how many are inside the system at the same time. Example: 1,000 requests per second, each taking 20 ms (0.02 s), means 1,000 × 0.02 = 20 requests in flight at any moment. This is an identity, not a design choice.
Logic skew: The batch and online implementations of one feature drift apart.
LSM-tree: Log-structured merge tree: buffer writes in memory, flush sorted runs, merge in the background.An LSM-tree (Log-Structured Merge tree) is a storage structure that never changes files in place. It collects writes in memory, writes them as sorted, immutable files, and later compacts (merges and rewrites) the files in the background to remove old versions and deleted entries. A tombstone is a marker that says "this key was deleted"; compaction removes the key and the marker. Compaction costs disk and CPU. Write amplification is the ratio of bytes the storage actually writes to bytes the application asked to write. A Bloom filter is a small structure that answers "definitely not in this file" or "maybe in this file", so reads can skip most files.
Memory-bound: Limited by data transfer from memory rather than by arithmetic.
Memory-bound versus compute-bound: Whether a computation is limited by waiting for data from memory or by the arithmetic itself. Below the machine's ridge point (FLOPs per byte loaded) it is memory-bound, and only reading fewer bytes makes it faster.A computation is memory-bound when it spends its time waiting for data to arrive from memory and the arithmetic units are mostly idle. It is compute-bound when the arithmetic itself is the limit. The roofline model says: divide the arithmetic operations (FLOPs, floating-point operations) by the bytes loaded; if the ratio is below a machine-specific threshold (the ridge point), the machine is memory-bound, and the only way to go faster is to read fewer bytes.
MIPS (maximum inner product search): Finding the vectors with the largest dot product with the query.
NDCG: A ranking-quality score that rewards putting highly relevant results near the top.
Normalization: Scaling vectors to length 1 so that cosine, dot product and Euclidean distance give the same ranking.1 other meaning →
nprobe: The IVF setting that controls how many cells are searched per query.
Offline store: The feature store layout optimized for large point-in-time-correct scans for training.
Online store: The feature store layout optimized for single-key, low-latency reads at serving time.
Partition pruning: Skipping partitions that cannot match the query's filter.A zone map stores the minimum and maximum of a column for each block of rows. If your range does not overlap a block's min/max, you skip the block without reading it. Partition pruning is the same idea at table level: a table is split into partitions by a key (for example by date), and the engine skips partitions that cannot match. Both rely on values being comparable, so that "min" and "max" mean something.
Percentile (p50, p99, p99.9): The value below which that fraction of observations fall.p50 is the median latency, p99 is the latency that 99% of requests beat. The slowest requests are the tail. When one request needs answers from many servers (a fan-out) it must wait for the slowest one. With 100 servers that each have a 1% chance of being slow, the chance that at least one is slow is 1 − 0.99¹⁰⁰ = 63%. So the user-facing tail is much worse than any single server's tail.
Percentiles do not compose: You cannot compute the overall p99 from the p99 of each shard. Averages and sums can be combined; percentiles cannot. This idea returns in several forms below: some summaries can be merged, and some cannot.
Plane sweep: Spatial join that sorts along one axis and keeps an active set of overlapping objects, the geometric sort-merge join.
Point-in-time correctness: Using, for each event, only the feature values known at that moment.
Post-filter: Retrieving top-k first, then dropping items that fail the predicate.
Postings list: The list of documents (and optionally positions) for one term.
PQ (product quantization): Compressing a vector into a short code, one codeword index per sub-vector.
Pre-filter: Applying the predicate first, then searching only the survivors.
Precision: Of the alarms that fired, the fraction that were real defects.Of the results returned, precision is the fraction that is correct. Of all the correct answers that exist, recall is the fraction you returned. If 10 items are truly relevant and you return 8 of them among 20 results, recall = 8/10 = 80% and precision = 8/20 = 40%.
Precision / recall: Fraction of returned results that are correct / fraction of correct answers that were returned.
Predicate: The condition in a query that decides which records qualify.The condition in a query that decides which records qualify. In WHERE price < 500 AND city = 'Pune', the predicate is price < 500 AND city = 'Pune'.
Pregel: A vertex-centred BSP model: vertices receive messages, update state and send messages in supersteps.
Property graph: Graph model where vertices and edges carry key-value properties.
Pruning: Skipping whole partitions or files using metadata.A columnar file format (such as Parquet) stores each column of a table separately, so a query that needs 5 of 200 columns can read only those 5. Partitioning splits a table into folders (for example, one per day). Pruning (or skipping) means not reading the parts a query cannot need. Selectivity is the fraction of data a query actually needs.
Quadtree: Spatial tree that splits each square into four equal squares.
R-tree: Spatial tree of nested, possibly overlapping bounding boxes.
Range: Everything between two values in a total order, such as dates in a month. A B-tree keeps keys sorted, so it reads a range as one contiguous stretch; a hash index can only find exact values.A total order means any two values can be compared and one comes first (numbers, dates, strings in alphabetical order). A range is "everything between A and B" in that order. An ordinary database index, the B-tree, keeps keys sorted so that "find everything between A and B" reads one contiguous stretch. A hash index answers only "is this exact value present?" in constant time.
RDF / triple store: Data model of (subject, predicate, object) triples with no other structure.
Recall: Of the real defects, the fraction that were caught.Of the results returned, precision is the fraction that is correct. Of all the correct answers that exist, recall is the fraction you returned. If 10 items are truly relevant and you return 8 of them among 20 results, recall = 8/10 = 80% and precision = 8/20 = 40%.
Reciprocal rank fusion (RRF): Combining result lists by summing 1/(constant + rank) for each item.
Reconciliation: Comparing data to a second independent source.Checking data against a second, independent source ("our revenue total equals the payment provider's total").
Recovery-time objective (RTO): How long until a service works again after a failure; for a derived index it is the rebuild time.
Refresh interval: How often buffered writes become a searchable segment.
Retention tier: A stored resolution level (raw, minute, hour, day) with its own keep period.
Reverse adjacency: A second copy of the graph indexed by incoming edges.
Right to be forgotten: A person's right to ask for their data to be deleted.
Rollup: A precomputed coarser aggregate of a time series.
Roofline model: Plot of achievable performance vs. arithmetic intensity, showing memory-bound and compute-bound regions.
Scatter-gather: Fanning a request out to N shards and waiting for their answers.To query a dataset split over many machines (shards), a coordinator sends the query to the shards (scatter) and merges the answers (gather). The query is finished only when the slowest shard answers, so the latency of the whole is the maximum over shards. Slow outliers (the "tail", measured by percentiles such as p99, the time that 99% of requests beat) therefore hurt more as you add shards.
Schema: The description of the shape of a record: fields, types, optionality.A description of the shape of a record: which fields (also called columns) exist, what type each has (integer, text, timestamp, ...), and which ones may be missing. Example: a shipment event has shipment_id (text), status (text), weight (number), occurred_at (timestamp).
Segment: A small immutable sorted file in an LSM-style or search index.A write-friendly storage design. New data is collected in memory, then written out as a small, sorted, never-modified file (a segment). Reads search all segments and combine the answers. A delete is recorded as a marker (a tombstone) instead of changing a file. In the background, compaction merges small segments into bigger ones and finally drops tombstoned data. The reason: changing a big sorted structure in place means many small random writes, which storage hardware handles badly. Merging sorted lists is cheap, so the design makes writes cheap and pays later.
Selectivity: Fraction of rows that pass a condition.The cardinality of an operator is the number of rows it produces. The selectivity of a condition is the fraction of rows that pass it. Example: a table has 1,000,000 rows and city = 'Pune' matches 20,000 of them. The selectivity is 20,000 / 1,000,000 = 0.02 (2%), and the cardinality of the filter output is 20,000.
Series: One distinct combination of metric name and label values.
Sketch (HyperLogLog): A small summary of a data set that answers one kind of question approximately.A quantile is a percentile: the value below which a given fraction of data falls (the median is the 50% quantile). A sketch is a small, fixed-size summary of a huge dataset that answers one kind of question approximately: for example a t-digest or KLL sketch answers quantile questions, HyperLogLog counts distinct values, and Space-Saving finds the most frequent values. A sketch is built in a single pass and uses bounded memory.
Snapshot id: An identifier for one committed version of a table.
Spatial join: A join whose predicate is a geometric relation rather than equality.
SRID: Spatial reference identifier attached to a geometry to name its CRS.
Stemming / stop word: Reducing words to a root form / a common word dropped from indexing or scoring.
Superstep: One round of a BSP computation.
t-digest / KLL: Mergeable sketches that estimate quantiles.
Tail latency: Latency of the slowest requests (p99 and beyond).To query a dataset split over many machines (shards), a coordinator sends the query to the shards (scatter) and merges the answers (gather). The query is finished only when the slowest shard answers, so the latency of the whole is the maximum over shards. Slow outliers (the "tail", measured by percentiles such as p99, the time that 99% of requests beat) therefore hurt more as you add shards.
Temporal split: A split that trains on earlier data and tests on later data.
Timing skew: The same logic sees different data because of what has arrived when it runs.
Tombstone: A marker record that says a key was deleted; must outlive every older copy.
Total order: An ordering in which any two values can be compared.A total order means any two values can be compared and one comes first (numbers, dates, strings in alphabetical order). A range is "everything between A and B" in that order. An ordinary database index, the B-tree, keeps keys sorted so that "find everything between A and B" reads one contiguous stretch. A hash index answers only "is this exact value present?" in constant time.
Training-serving skew: Any difference between the feature values seen in training and in serving.
Vectorized UDF: A user-defined function that receives a block of columnar rows at once, amortizing the call cost.
Vertex-cut partitioning: Assigning each edge to a machine and replicating high-degree vertices across machines.
Write amplification: Bytes written per byte of user data.If you write 1 MB of data and the system physically writes 10 MB (because of rewriting and merging), the write amplification is 10.
Zero-copy columnar interchange: A shared in-memory column layout that lets two runtimes exchange data by pointer, with no serialization.
Zipf distribution: A distribution where the r-th most common value has frequency about 1/r of the top one.In many workloads a few items are requested very often and most items are requested rarely. Zipf's law says the item of popularity rank i (rank 1 is the most popular) is requested with frequency proportional to i^(−α). The number α (alpha) is the popularity exponent. A larger α means a steeper concentration on the top items.
Zone map: Per-file min and max values of a column, used to skip files.A zone map stores the minimum and maximum of a column for each block of rows. If your range does not overlap a block's min/max, you skip the block without reading it. Partition pruning is the same idea at table level: a table is split into partitions by a key (for example by date), and the engine skips partitions that cannot match. Both rely on values being comparable, so that "min" and "max" mean something.