Choose an execution model
A workload must run: a SQL query, a nightly batch job, an iterative graph algorithm, a stream of per-device events. The execution model decides how work is split, how data moves between steps, and what a failure costs: a whole restart, one task, or nothing. Which model fits this workload?
You will end with an execution model, and the arithmetic that justifies it, including how a failure is recovered.
Decide
Does the data fit in one machine's memory or local storage?
Check before assuming a cluster: most 'we need a cluster' decisions fail this question.
YesOne machine
NoGo to 2
Why
Across machines, every handoff costs milliseconds and can fail, so the cluster models exist to manage that. On one machine the handoff is a memory access, and the choices that matter are different and cheaper.
What shape is the workload?
RoundsBulk-synchronous rounds (BSP)
Per-key stateActors
Bursty, shortServerless functions
DataflowGo to 3
Why
- Rounds: iterative algorithms where every step needs a consistent view of the previous round (PageRank, graph algorithms).
- Per-key state: long-lived state per key, accessed irregularly (a device, a session, an account).
- Bursty, short: jobs that are short-lived and narrow, and arrive in bursts.
- Dataflow: a query or batch job, a graph of operators passing data along.
How likely is a failure during one run?
Multiply tasks × hours each task runs × failures per task-hour. 10,000 tasks of 0.1 hour on spot machines failing 10⁻³ per hour gives exactly 1.
Well under 1Streamed (pipelined) dataflow
Near or above 1Materialized dataflow (DAG of tasks)
Why
If any failure restarts the whole query, a run succeeds only when nothing fails: probability about e^(−product). At a product of 1, that needs about 2.7 attempts on average. Writing each stage's output down makes the task, not the query, the unit of failure.
Where you can land
One machine
Use it when the data fits in one machine's memory or local storage.
Then decide inside the machine
Direction: push or pull
A single-threaded pipeline: push, which keeps values in registers. Across threads you build flow control either way: pull if you need cheap LIMIT and cancellation, push if you need cheap fan-out to several consumers and code generation.
Granularity: tuples or batches
Below about 1,000 tuples per pipeline, one tuple at a time is free and simplest. Above that, process batches sized to fit the CPU's L1 cache: L1 size ÷ width of the live row.
Compiled or interpreted
Compute-heavy queries with many expressions favour compiling the query to machine code. Memory-bound ones with large hash tables favour vectorized operators that keep many cache misses in flight. Then ask whether you can afford a compiler in the process and the loss of profiling tools: for most teams that decides it, not the performance difference.
Compute the break-even volume
Whichever you chose: break-even = setup cost ÷ saving per tuple. If your workload straddles it: you will end up building tiers (interpret first, compile when hot). Plan for it instead of discovering it.
Streamed (pipelined) dataflow
Use it when the data needs a cluster and a failure during a run is unlikely.
Operators pass data directly to the next stage as it is produced.
What it costs: a failure restarts the whole query; intermediate data must stay in memory.
Materialized dataflow (DAG of tasks)
Use it when the data needs a cluster and a failure during a run is likely.
Each stage writes its output to storage before the next reads it, so a failed task reruns alone.
What it costs: extra I/O and latency at every stage boundary.
Bulk-synchronous rounds (BSP)
Use it when the algorithm iterates and each round needs a consistent view of the last.
Every worker computes, exchanges messages, then waits at a barrier for all others before the next round.
Before you build it: each round waits for its slowest worker, so multiply the straggler delay by the number of rounds before believing an estimate. If that is unacceptable and the algorithm tolerates stale values: run asynchronously, and pay in determinism.
Actors
Use it when state lives per key for a long time and is accessed irregularly.
Each key's state lives in one actor that processes its messages one at a time.
Before you build it: bound every actor's mailbox from day one. An unbounded mailbox is an unbounded queue.
Serverless functions
Use it when work is bursty, short and narrow.
Before you build it: functions exchange data through object storage, which bills per request. Check bytes per exchanged object, total bytes ÷ (producers × consumers), against object storage's efficient size of about 4 MB, and check the request charges against the compute bill. A pilot that is fast can still be dominated by storage requests.