What orchestration forbids, and the price of each escape
What the layer forbids
Six impossibilities. Everything above is an accommodation to one of them.
- You cannot verify success. You only receive a claim from the party that may have failed. Independent evidence must be constructed on the output.
- You cannot undo effects. No rollback exists across systems you do not own. So forward-only recovery through idempotent re-execution is the only mechanism, and it must be designed into the output layout.
- You cannot tell a dead task from a slow one. So you cannot make relaunch safe by detection, only by fencing. Fencing needs the sink's cooperation, which is usually absent.
- You cannot detect completeness from arrivals. Completeness is an absence. You need a sentinel, a count or a timeout. There is no fourth option.
- You cannot detect your own absence. Liveness needs an observer outside the system responsible for the liveness.
- You cannot verify a declared dependency. The graph is a human claim about ordering, and the undeclared edges are the ones that produce incidents.
The escapes and their prices
| Escape | Buys | Costs |
|---|---|---|
| Idempotent overwrite by interval key | Safe retry | Write amplification at partition granularity; output layout must match the schedule |
| Write-audit-publish | Atomic visible transition, independent verification | A staging copy plus the assertion run time |
| Fencing tokens | Safe relaunch under uncertainty | Needs a cooperating sink; most sinks do not cooperate |
| Sentinels / manifests | A usable completeness signal | Another unverifiable producer claim; lost on a crash between data and marker |
| Deferrable sensors | Waiting without holding slots | A shared poller as new infrastructure and a new single point of failure |
| Dataset declarations | Decouples consumer from producer identity | Coarser than the real dependency; needs an owned namespace |
| External dead-man's switch | Detects the run that never started | A second system, with its own availability |
| depends_on_past | Correct cumulative semantics | Sequential backfill duration; one failure blocks all successors |
| Durable execution | Fine-grained retry, real control flow, durable human waits | Enforced determinism; inter-step state must be small; explicit versioning burden |
| Choreography | Removes the central coupling point | Global state must be rebuilt from traces, after the fact, approximately |
The single sentence to keep: orchestration is the discipline of making unreliable, unobservable, irreversible side effects behave like a pure function of a time interval. Every mechanism in the layer either establishes that purity or compensates for its absence.