Glossary

64 terms, in the sense this mechanism uses them. The platform glossary has every term across all mechanisms.

Admission control: Declining new work before any resources are committed to it.
AIMD: Additive increase, multiplicative decrease: a rate-control rule that grows slowly and cuts sharply when congestion appears.
AND-job / OR-job: A job that needs all parts to survive (AND) versus any part (OR).
Anti-affinity: A human-supplied rule that keeps certain tasks (for example replicas) apart.
Autoscaling: A control loop that adds or removes capacity; it has a delay L = detect + decide + provision + warm up.
AZ (Availability Zone): An isolated datacenter (or group) inside a cloud region.A rack is a cabinet of servers that share one network switch and one power feed. An AZ is a separate datacenter building (or group) inside a cloud region. Things in the same rack or AZ tend to fail together.
Backfill: Running short jobs in idle gaps before a reserved large job starts.1 other meaning →
Bin-packing: Choosing groups of small files to rewrite as files near the target size.
Capacity scheduling: Each pool has a guaranteed floor and may borrow idle capacity above it.
Checkpoint: A saved snapshot of all operator state plus source offsets.1 other meaning →A saved snapshot of a job's state plus its position in the input, written to durable storage (storage that survives a crash). After a crash the job loads the last checkpoint and continues from there.
Classification (at admission): Guessing a query's size or kind to route it to a lane or engine.
Cold start: The billed time to start a machine, runtime and caches before useful work.
Concurrency limit: A cap on the number of requests in flight.
Critical path (span, T∞): The longest chain of dependent tasks in a DAG; the lower bound of makespan.
DAG: Directed acyclic graph: tasks joined by arrows with no loops.Directed acyclic graph: boxes (tasks) joined by arrows (dependencies) with no loops. "Directed" means each arrow has a direction. "Acyclic" means you cannot follow the arrows and come back to where you started. An arrow A → B means "B may start after A finishes."
Deadline: The time by which a result is still wanted.
Delay scheduling: Waiting briefly for a data-local slot instead of running remotely at once.
Descheduler: A background process that evicts tasks whose placement has become invalid.
DRF: Dominant Resource Fairness: equalize each tenant's largest resource share.
EDF: Earliest Deadline First: run the task with the nearest deadline.
FIFO: First in, first out.
Flow time: The time from submitting a job to its finish: wait time in the queue plus run time. The time users actually feel; also called response or completion time.Wait time is how long a job sits in the queue. Run time (also called service time or job size) is how long it actually works. Flow time (also called response time or completion time) is wait plus run: the time from submission to finish. Users feel flow time.
Gang scheduling: Allocating all tasks of a job at the same moment, or none.
Goodput: Completed work whose result is still wanted.Throughput is work completed per second. Goodput is work completed per second whose result is still wanted. If a client has already given up (timed out), finishing its request adds throughput but no goodput.
Graham's anomaly: A greedy schedule can get slower when resources are added.
Head-of-line blocking: An item at the front of a queue blocking all items behind it.
Heavy tail: A distribution in which a small fraction of items carry most of the total size.A distribution with a few extremely large values (a phone offline for a week) far above the typical ones.
Heavy-tailed sizes: In many real workloads most jobs are tiny and a few are enormous. A small fraction of jobs consumes most of the resource. Real query and batch workloads usually look like this.
Idempotent: Doing it twice has the same effect as doing it once.An operation is idempotent if doing it twice has the same effect as doing it once. "Set the order status to SHIPPED" is idempotent. "Add 1 to the stock counter" is not. Idempotence is the main tool for living with duplicates.
Job: A unit of work a user submits (a query, a nightly pipeline, a training run), usually split into many tasks that run on different machines, each in a slot: a fixed share of a machine.A job is a unit of work a user submits (a SQL query, a nightly pipeline, a training run). A job is often split into many tasks, which are small pieces that can run on different machines. A slot is a fixed share of a machine (for example "1 CPU core and 4 GB of memory") that one task can occupy.
Knee: The arrival rate where latency starts to rise sharply.1 other meaning →
Lane (reserved): Capacity held for one class of work, such as short queries.
Little's Law: L = λW: average items in a system = arrival rate × average time in the system.Throughput is work finished per second. Latency is how long one operation takes. Little's Law links them to the number of operations in progress: L = λ × W, where λ is the arrival rate (per second), W is the average time one operation spends in the system, and L is how many are inside the system at the same time. Example: 1,000 requests per second, each taking 20 ms (0.02 s), means 1,000 × 0.02 = 20 requests in flight at any moment. This is an identity, not a design choice.
Makespan: Total time for a whole job to finish.
Max-min fairness: Maximize the smallest allocation, then repeat on what is left.
Mean: The sum of values divided by their count.The mean is the average. The tail is the slowest few requests. "p99" means the value that 99% of requests are faster than, so 1% are slower. Averages hide the tail. A policy can improve the mean and make the tail worse.
MLFQ: Multi-level feedback queue: jobs sink to lower priority as they use CPU.
Multiplexing gain: Capacity saved by sharing, about √N for N independent tenants.
Noisy neighbor: One tenant slowing others through resources outside the quota.
Overcommit ratio: How much more is promised (requested) than the machine physically has.
Preemption: Stopping a running task so others can use its resources.Stopping a running task so that something else can use the resource.
Priority inversion: A high-priority task effectively runs at lower priority because it waits for a resource held by a lower one.
Queue: A line of work waiting for its turn, like people waiting at a bank counter. A scheduler is the part of a system that decides who is served next.1 other meaning →
Quota: A limit on a resource that can be named and attributed to a tenant.
Rack: A rack is a cabinet of servers that share one network switch and one power feed. An AZ is a separate datacenter building (or group) inside a cloud region. Things in the same rack or AZ tend to fail together.
Result governor: A limit on result size that protects the client and network.
Rivalrous resource: A resource is rivalrous if two users cannot use it at the same instant. One CPU core can run one task at a time. If you hold it, I do not have it. This is the root cause of everything here.
Run time: How long a job actually works once it starts, also called service time or job size. Flow time is wait time plus run time.Wait time is how long a job sits in the queue. Run time (also called service time or job size) is how long it actually works. Flow time (also called response time or completion time) is wait plus run: the time from submission to finish. Users feel flow time.
Runaway killer: A rule that stops a task whose cost exceeds a cap.
Shuffle: Moving rows between stages so that data is partitioned by a different key.Two stages of a parallel query often need the data grouped by different keys. The shuffle moves every row from the machine that produced it to the machine that needs it. It is the most expensive data movement in a distributed engine.
Shuffle sharding: Assigning each tenant a random subset of k nodes out of n so that few tenants share the same subset.
Slot: A fixed unit of capacity used for static admission.A job is a unit of work a user submits (a SQL query, a nightly pipeline, a training run). A job is often split into many tasks, which are small pieces that can run on different machines. A slot is a fixed share of a machine (for example "1 CPU core and 4 GB of memory") that one task can occupy.
Spot: The vendor's spare machines, 60–90% below on-demand, with short-notice interruption.
SRPT: Shortest remaining processing time first.
Starvation: One class of work is never served while the system is otherwise busy.
Stranded resource: Capacity that cannot be used because another resource is exhausted.
Tail (a log): The slowest requests, such as those above p99.p50 is the median latency, p99 is the latency that 99% of requests beat. The slowest requests are the tail. When one request needs answers from many servers (a fan-out) it must wait for the slowest one. With 100 servers that each have a 1% chance of being slow, the chance that at least one is slow is 1 − 0.99¹⁰⁰ = 63%. So the user-facing tail is much worse than any single server's tail.
Task: One step of a workflow.A workflow (also pipeline or DAG run) is a set of steps that must happen in a certain order. A task is one step: "run this SQL", "start this Spark job", "call this API". The orchestrator does not do the work itself. It tells another system to do it.
Throughput: Work completed per unit of time.Throughput is work finished per second. Latency is how long one operation takes. Little's Law links them to the number of operations in progress: L = λ × W, where λ is the arrival rate (per second), W is the average time one operation spends in the system, and L is how many are inside the system at the same time. Example: 1,000 requests per second, each taking 20 ms (0.02 s), means 1,000 × 0.02 = 20 requests in flight at any moment. This is an identity, not a design choice.
Utilization (ρ): The fraction of time a resource is busy.Utilization (ρ, "rho") is the fraction of time a server is busy. For random arrivals, average response time grows like S ÷ (1 − ρ), where S is the service time: 2×S at 50%, 5×S at 80%, 10×S at 90%, 100×S at 99%. Queues are caused by variability meeting load, and the curve has a wall at 100%. The last few percent of utilization cost more than the first eighty.
Variance: How much results differ from one another when the same thing is repeated. A difference smaller than the variance of repeated runs may be noise.Variance is how much results differ from each other when you repeat the same thing. Noise is the part of a result that comes from chance and not from your change. A confidence interval (CI) is a range such as "12% faster, 95% CI [7%, 17%]", meaning "the true improvement is probably between 7% and 17%."
Wait time: How long a job sits in the queue before it starts running. Flow time is wait time plus run time.Wait time is how long a job sits in the queue. Run time (also called service time or job size) is how long it actually works. Flow time (also called response time or completion time) is wait plus run: the time from submission to finish. Users feel flow time.
Walltime: A declared maximum run time for a job.
Work-conserving: A policy that never leaves a machine idle while work is waiting.