Simplicity vs. adaptivity
Where you meet it: static plans vs. adaptive execution.
What it is
Static versus dynamic scheduling is a choice between simplicity and adaptivity. Static scheduling fixes the assignment of tasks to workers before execution, using the planner's estimates. What it buys is co-optimization: all decisions are made together, with global knowledge of the plan. You can place tasks near their data, gang-schedule stages that must run at the same time, reserve memory coherently, and pay zero runtime overhead.
From Execution Engine Runtime › Stealing and speculation
Branching defeats static analysis of the graph. Before the run you can no longer compute the critical path (the longest chain of dependent tasks), the expected resource use, or the set of outputs that will be produced. Anything that wanted to plan against the DAG shape (cost estimates, capacity reservation, DAG-aware scheduling) degrades to a guess. Simplicity versus adaptivity has a precise price here: adaptivity costs the plannability of everything upstream of the decision.
From Orchestration & Workflow › Branching
Where it appears
In 6 mechanisms. Each says what the mechanism is about, then how this trade-off shows up there, in its own words.
- Encoding, Serialization & Compression
What do we turn our values into when we turn them into bytes, and what does that choice cost?
Per-value adaptivity costs sequential decoding.
Read it inVarints and order-preserving encoding
- Query Planning & Optimization
How does a system choose a plan for a question when the facts that decide the best plan are unknown until the plan has run?
You cannot have both plan determinism and plan adaptivity as defaults.
Read it inWhat execution decisions forbid
- Execution Engine Runtime
Once a query plan is chosen, why does the machine still sit idle, and what does each mechanism of a query runtime pay to buy that idle time back?
Static versus dynamic scheduling is a choice between simplicity and adaptivity.
- Memory & Resource Management
Who is allowed to hold how much memory, for how long, and what happens to the work when the answer is less than it asked for?
It is adopted because its hit path does not serialize, and the hit rate it gives up is smaller than the throughput that LRU's bookkeeping costs.
Read it inClock: chosen for its hit path, not its hit rateLRU-K, 2Q and ARC
- Fault Tolerance & Delivery Semantics
What is the cheapest thing that can be safely done twice?
Where it is expensive or lossy (a write primary behind asynchronous replication, where a spurious promotion can lose committed data and cause a split brain, two machines both acting as primary), mature teams deliberately keep a human in the loop.
Read it inFailover: automatic, manual, fenced
- Scheduling & Workload Management
Which work runs now, which work waits, and who pays for the waiting?
> When capacity is unknown, you find it by increasing until latency bends, and you back off faster than you advanced.
Read it inConcurrency limits and slots
Related
Made on the same pages most often.