Handle overload
Work is arriving faster than a service or consumer can process it: latency climbs, queues grow, and timeouts start. Something must give: the work can wait, the source can slow down, new work can be refused, or some work can be dropped. Which response fits this overload?
You will end with a response to overload, what it gives up, and a full-policy for every buffer.
Decide
Is one specific dependency failing or slow, rather than the whole system being overloaded?
A partner API returning 429 and 503 errors, while everything else is healthy.
Why
Calls to a failing dependency tie up workers and connections that the rest of the system needs. Stopping those calls quickly protects everything else.
Is it a burst, or does arrival stay above service rate?
800 events a second of capacity against a 2-hour peak of 2,000 is sustained; a few seconds of spike is a burst.
BurstBuffer, sized for the burst
SustainedGo to 3
Why
A buffer only moves work in time. While arrival exceeds service, a buffer of capacity C lasts C ÷ (arrival rate − service rate) and then fills. Against sustained overload, no buffer is big enough.
Can the source be slowed down without breaking anything?
A batch job reading from a log can pause; users clicking checkout cannot.
YesBackpressure
NoGo to 4
Why
Backpressure pushes the overload back to where it started, which is the only response that loses nothing. It works only if the source can wait.
Can you refuse work before doing any of it?
Why
Refusing at the door costs nothing. Accepting work and dropping it later wastes what was already spent, so it is the last resort, and then what you drop is the whole design.
Where you can land
Circuit breaker
Use it when one dependency is failing, not the whole system.
After enough failures, stop calling the dependency for a while and use a fallback; then let a few trial calls through to test whether it has recovered.
Before you build it: define the fallback, and add random jitter to recovery so every client does not retry at the same moment.
Buffer, sized for the burst
Use it when the overload is temporary.
Before you build it: size it in seconds, not bytes: how long it survives at the expected excess. Decide what happens when it is full.
Backpressure
Use it when overload is sustained and the source can be slowed safely.
The consumer signals the producer to slow down. Prefer pull (consumers fetch when ready); use credits (a producer may send only as many records as the consumer has granted) when you need several records in flight.
What it costs: it propagates. Slowing one stage slows everything upstream of it, which is its power and its danger.
Admission control
Use it when overload is sustained, the source cannot wait, and work can be refused up front.
Refuse new requests when the number in progress reaches a limit, before any work is done on them.
What it costs: refused callers must retry or give up; give them a clear signal so they back off.
Load shedding
Use it when everything must be accepted, and some of it dropped later.
Before you build it
Decide what to drop first
In rough order of value: work whose deadline has passed (it is useless anyway); low-criticality work (backfills, recommendations, prefetch) before checkout; expensive requests before cheap ones; the noisy tenant before the well-behaved one; superseded values (old prices, old positions); and finally a recorded sample, so analytics can correct for it.
Pass deadlines along first
Dropping by deadline needs the deadline to travel with the request. If deadlines are not propagated: add that before anything else.
Whichever you land on
Stop one flow from starving another
If one tenant or workload can use up shared workers, connections or memory, partition them (bulkheads). Hard partitions cost the sum of each partition's peak instead of the peak of the total: three tenants peaking at 100 each, never together, need 300 units isolated against 150 shared. The middle ground: a guaranteed floor per partition plus a shared burst pool.
Give every buffer a full-policy
For every buffer in the path, name what happens when it is full: block, drop the newest, drop the oldest, or fail. If any answer is "unbounded" or "I do not know": that is your next bug.