Skip to content
The Data EngineeringThe Data EngineeringBeta
About
Main content

Resource-exhaustion cascade

How fast does a slow dependency exhaust a shared pool, and what do bulkheads cost?

This is the mechanism that couples things with no call-graph relationship at all. It is the most common form of "why did an unrelated feature break?"

The coupling agent is a shared fungible resource: a thread pool, a connection pool, file descriptors, memory or a semaphore (a counter that limits how many tasks may use something). A request that is blocked on a slow dependency D holds one unit of a resource that serves all dependencies. So D's becomes everyone's .

The timing can be computed

Use Little's Law (L = λ × W) for the number of concurrent requests held:

  • Normal latency 10 ms at 100 req/s: L = 100 × 0.01 = 1. One thread is enough.
  • D degrades to 10 s (a 1,000× change in latency, nothing unusual during an incident): L = 100 × 10 = 1,000 threads needed.
  • The time to exhaust a pool of size P is about P/λ. A 200-thread pool at 100 req/s is gone in 2 seconds.

Pool exhaustion is therefore not a risk to be managed. It is arithmetically certain for any finite pool facing any sufficiently slow dependency. The only variables are how fast it happens and how much else goes down with it. This is exactly Selin's booking service: the recommendations call took the shared thread pool, and every other call (pricing, maps, payments) found no thread.

The regime flip in the shared pool

A shared pool is the right design under normal conditions, because of statistical : one pool sized for the combined peak is much smaller than k separate pools each sized for its own peak (statistical multiplexing). The same sharing that saves resources in the common case is what transmits the failure. This is the regime-flip principle again, and it is why exist. A bulkhead is a separate pool for each dependency, like the watertight compartments of a ship, so a leak in one does not sink the others.

// Bulkhead: each dependency gets its own small pool; a slow one exhausts only its own.
ExecutorService recsPool    = Executors.newFixedThreadPool(20);
ExecutorService pricingPool = Executors.newFixedThreadPool(80);
ExecutorService mapsPool    = Executors.newFixedThreadPool(60);

What gets partitioned

  • Thread pools or concurrency budgets, per dependency (the case above).
  • Connection pools, per downstream.
  • Queues or partitions, per tenant or per key. This prevents caused by one bad actor or one poison record (a record that makes the consumer fail every time).
  • Cells or shards. The strongest form: the full stack copied per cell, the one cell (cells).
  • Separate clusters for batch and interactive work. This is the data-engineering version. If a backfill and a dashboard query share one warehouse, the backfill eats the dashboard's capacity. Resource groups, separate warehouses and pools exist for this.

The cost: isolation versus utilization

Partitioned resources cannot be shared, so each partition must be sized for its own peak and sits idle the rest of the time. The total provisioned capacity is Σ peak(i) (the sum of each partition's own peak) instead of peak(Σ) (the peak of the combined load). Since peaks rarely happen at the same time, the difference can be a large multiple.

The mitigation is hierarchical: a guaranteed floor per partition plus a shared burst pool. Each tenant is isolated up to its reservation, can burst into shared capacity when it is available, and can never push another tenant below its floor. It adds complexity and recovers most of the . This is the design you want when plain partitioning turns out to be too expensive.