Interview questions
Feedback loops
1What is a feedback loop in a distributed system, and why does it change how you reason about load?
The load clients offer depends on the latency they see, because they retry and time out, and capacity in turn depends on the load. Load is no longer an independent input, so you need a control view, with gain and delay, not just a queueing view with a fixed arrival rate.
2What decides whether a disturbance dies out or grows?
The loop gain: how many extra requests each request causes. Below 1, a disturbance dies out; above 1, it grows until some limit, such as a full queue, is hit.
Read moreGain decides
3How large can retry amplification get across a chain of services?
It multiplies at each tier: (1 + r)^d for r retries per tier and d tiers. With 3 retries per tier over 4 tiers, the deepest tier can see 4⁴ = 256 times its normal load.
Read moreComposing over a call chain
4What is a retry budget, and what does it bound?
A rule that allows a retry only while recent retries are below a fraction, say 10%, of recent successes. It bounds amplification to about 1.1×, however deep the call chain.
Read moreThe structural fix: a retry budget
5What is the difference between a thundering herd and a cache stampede?
A thundering herd is many clients acting at the same moment, triggered by something like a clock tick or a restart. A cache stampede is many requests missing the same expired cache key and all hitting the database together.
Read moreThundering herdCache stampede
6Does exponential backoff alone fix synchronized retries?
No. Clients that failed together keep retrying at the same instants, just less often. Random jitter is what spreads them apart.
7What is congestion collapse, and how does it show up on a graph?
Throughput stays high while goodput, useful completed work, falls, because effort goes into requests whose clients have already given up.
Read moreCongestion collapse
8Why can a small, bounded queue be better than a large one?
It turns a long wait into a fast error, which clients can handle and which costs no downstream capacity. A large queue only hides overload and adds delay (bufferbloat).
9Why does routing to the least-loaded server sometimes oscillate?
The load information is slightly old, so every router picks the same server at the same moment and overloads it. The power of two choices, sampling two random servers and picking the less loaded, avoids herding.
Metastable failure
10What is a metastable failure?
A failure where the system stays broken after its trigger is gone, because the degraded state sustains itself. It can happen when load sits in the band C / k < λ < C, for capacity C and an amplification factor k in the degraded state: below capacity when healthy, but above what the degraded system can serve.
Read moreThe formal condition
11In a metastable failure, should you fix the trigger or the sustaining effect?
The sustaining effect. Removing the trigger, such as a 3-second stall, does not help once the feedback loop is holding the system down.
Read moreTrigger versus sustaining effect
Cascading failure
12A service has 30 hard dependencies, each 99.9% available. What is its availability ceiling?
About 0.999³⁰ ≈ 97.0%, whatever its own code does.
Read moreDependency cascade
13What is a cell, and why use cells?
An independent, complete copy of a service that serves a subset of customers, so one failure affects only that subset. Outages become smaller, though not rarer.
Read moreBlast radius, failure domains and cellsThe honest arithmetic
14What is static stability?
A system keeps working on its last known good state when its control plane is down, because the data plane does not need the control plane to serve normal requests.
Read moreStatic stability
15What is a bulkhead, and what does it cost?
A partition of resources, such as separate thread pools, queues or cells, so one failing part cannot consume capacity that others need. It costs utilization: you must provision the sum of each part's peak instead of the peak of the total. Reduce the cost with a guaranteed floor for each part plus a shared pool for bursts.
Read moreThe cost: isolation versus utilizationThe regime flip in the shared pool
Gray failure
16Why is a node that fails slowly worse than one that stops?
A dead node is detected and removed, and costs 1 / N of capacity. A slow node stays in rotation and can delay every request that touches it.
17What is differential observability?
Different observers see different health for the same component: its health endpoint says green while clients see timeouts. To find it, tabulate latency by client and by server.
Read moreDifferential observability
Contention
18What is a lock convoy, and why does it persist?
Threads queue behind a lock holder that was descheduled, then move in lockstep. Each release resynchronizes them, so the convoy outlives whatever started it.
Read moreConvoy effect
Death spirals
19Why does garbage-collection cost grow so fast as the heap fills?
The cost per byte freed is about k ρ / (1 − ρ), where ρ is how full the heap is. And by Little's Law, slower responses mean more requests in flight, so occupancy rises with latency.
Read moreGC death spiral
20What write rate can an LSM storage engine sustain?
Disk bandwidth divided by write amplification: for example 2 GB/s ÷ 50 = 40 MB/s. Writing faster than that builds compaction debt that compounds.
Read moreLSM compaction debt
21Why is autoscaling not a defence against overload?
Its reaction time is minutes, while overload plays out in seconds; a controller that slow oscillates. And new instances start cold, so at first they reduce capacity.
Read moreAutoscaling oscillationAdding capacity first removes capacity
22After an outage of length T, how long does it take to recover at utilization ρ?
About T × ρ / (1 − ρ): as long as the outage itself at 50% utilization, 4 times as long at 80%, 9 times at 90%.
Read moreBacklog burst and replay storms
Overload defences
23Why does a circuit breaker need jitter and two thresholds?
Every client runs the same breaker logic, so without jitter they all open and close together. And a recovering service is cold, so traffic must ramp back up rather than return all at once.
24When should hedged requests be turned off?
Under overload, where duplicates add load, and against stragglers that are slow for a deterministic reason, where a duplicate is just as slow.
Read moreHedged requests
25What states does a circuit breaker have, and what are the common mistakes in using one?
Closed (calls pass), open (calls fail fast) and half-open (a few trial calls test recovery). Common mistakes:
- counting errors instead of using an error rate;
- opening all or nothing;
- one breaker covering too much;
- no fallback when open;
- all instances recovering at the same moment;
- tripping on client errors (4xx).
It also needs a timeout, or a hanging dependency never registers as a failure.
Read moreCircuit breaker
Six questions for any design
26Which questions find failure dynamics in a design before they show up in production?
Six, in order: where is the loop (its gain and its delay); what is k, the amplification in degraded mode, which sets the vulnerable band C/k < λ < C and can be measured while the system is healthy; what is shared, which caps availability at the least available common component; who observes health, and are they inside the boundary they certify; what work is deferred, and does its cost rise with the backlog; and what does recovery cost, and is anything provisioned for it.
Read moreThe transferable diagnostic