Skip to content
About
Main content

Scenarios

Longer problems, like those in senior system design interviews. A strong answer estimates with numbers first, names the mechanism, then recommends a fix and says what it costs.

Try each one before you open the answer.
1A wallet service reads a balance, checks it in Java, then writes the new balance. About 300 conflicting operations per second hit hot accounts, and you see rare lost updates. Diagnose and fix.Show the answer

Estimate

With 40 ms of application time and , about operations overlap each , which is why it happens at all. A single SQL statement taking 40 µs would give 300 × 0.00004 = 0.012, a thousandfold reduction.

Mechanism

Read Committed allows the lost-update shape: one read–write edge plus one write–write edge.

Fix, cheapest first

  1. SET balance = balance - x, guarded by CHECK (balance >= 0): it , and the rule is a single-row constraint.
  2. Otherwise, a version column and a retry loop.
  3. Otherwise, SELECT … FOR UPDATE, accepting blocking and the risk of deadlock.

Failure modes

A retry loop needs an owner and a limit; must always be taken in the same order.

Recommend

The commutative update.

Read moreRead Committed: the home of lost updatesThe anomaly rate is Little's Law again

2Two operations each check that a room's total capacity would not be exceeded, then insert a booking. Occasionally the room is overbooked. The database runs at Repeatable Read. What is going on, and what are the options?Show the answer

Diagnose

: read a set (the bookings), check a rule over it, write into it. In PostgreSQL, is , whose conflict check sees nothing, because the two transactions inserted different rows.

Options

  1. Run those transactions at and retry on serialization failure (error 40001). Cost: some false and memory for tracking reads.
  2. Make the conflict concrete by updating a single room row in both. Cost: false conflicts, and it silently stops working when a new code path forgets the update.
  3. A unique constraint, if the rule can be expressed as one: enforced per row, and free.

Recommend

Option 1 for a few code paths, since this analysis does not compose across paths. On MySQL's InnoDB the locking behaviour differs, so test there.

Read moreWrite skew: one generative shapeRepeatable Read: three promises

3A new engineer says: 'We moved to a database that is serializable, so users will never see old data.' How do you respond?Show the answer

Correct the claim

Serializability limits which can happen, not how recent a read is. A read-only transaction may be ordered far in the past, and a may serve stale data. Recency is the other axis: single-object consistency. combines the two, and costs real-time coordination.

Practical advice

After a write, route the user's reads to the using a token, or use a system that offers , and measure .

Size it

With 667 operations per second and 2 s of lag, about 1,333 operations sit inside the stale window at any moment.

Read moreTwo hierarchies that must not be mergedExternal consistency

4Three services each use serializable transactions. A saga that spans all three shows inconsistent totals. Why?Show the answer

Diagnose

Serializability does not compose: independently serializable systems put together need not be serializable. Linearizable objects would compose, but transactions across services do not.

Options

Keep the in a single database; add a coordination protocol; or redesign so each invariant lives inside one service and uses commutative operations.

Say plainly

That each service is transactional tells you nothing about the pipeline as a whole.

Read moreComposability decides architecture

5Choose isolation levels for a ledger with three kinds of work: transfers, a nightly report, and a rule that a joint account must keep a combined minimum. Justify each.Show the answer

Transfers

Restate them as commutative deltas on rows with CHECK (balance >= 0); then any level works.

Nightly report

It needs one consistent to avoid , so snapshot isolation (or READ ONLY DEFERRABLE serializable). Read Committed would show totals that never existed.

Joint-account rule

A rule over a set of rows, so snapshot isolation is disqualified (write skew). Use SERIALIZABLE on that path and measure the abort rate, or a shared guard row.

Also

A vendor's name for a level must be tested, for example with a three-transaction test, not trusted.

Read moreWhat transactional isolation forbids

6After adding a read replica and a Redis cache, support tickets mention stale balances and missing history. Explain, and propose a plan.Show the answer

Explain

Each is a new place that can observe the data. The replica lags by 20 ms to 2 s; the cache serves values until their TTL expires. Expected anomalies ≈ rate of conflicting operations × window.

Plan

, through a session token or by routing a user's reads to the primary for a short time after they write. Bound staleness and monitor lag. Keep serializable checks on the primary. Document which screens tolerate stale data.

Trade-off

More load on the primary.

Failure mode

Tokens lost when a request passes through a component that does not forward them.

Read moreThree things change completelySession guarantees

7A wallet app expands from Frankfurt to Singapore, 80 ms apart one way. Product wants the balance screen to load in under 50 ms in Singapore, and transfers to stay correct. Design the read and write paths.Show the answer

Start from the arithmetic

Any linearizable or sequentially consistent path pays , so a 50 ms read in Singapore cannot be both linearizable and served from Frankfurt.

Options

  1. A single in Frankfurt for transfers (writes from Singapore take a 160 ms round trip: rare and acceptable), and the balance screen served from a Singapore replica with a stated bound, for example at most 2 s stale, with an alarm.
  2. Reads from the leader for the sender after a transfer, using a session token, so senders always see their own transfer.
  3. Lease-based local reads, if the product truly needs linearizable local reads, accepting an assumption about clocks that cannot be verified.

Estimate the window

At 667 conflicting operations per second and 2 s of lag, about 1,300 operations are inside the window.

Failure modes

The token is dropped at a load balancer (silently stale); the lag exceeds the bound (alarm, and fall back to the primary).

Recommend

Option 1 plus tokens, and never promise linearizable local reads in Singapore.

Read moreThe latency floorSession guaranteesFour ways to buy a fixed total

8A campaign gives a bonus to the first 5,000 customers across three regions. The limit must never be exceeded, and each region must keep accepting sign-ups during a network partition. Design it.Show the answer

Classify

"At most 5,000" is an upper bound that sign-ups push towards, so it is not invariant-confluent: coordination is required somewhere.

Design

Escrow. Split the 5,000 into regional quotas, for example 2,500, 1,500 and 1,000 by expected volume. Each region issues bonuses from its own quota with no cross-region call, and asks for more when it falls below a threshold.

Estimate

Coordination rate ≈ consumption ÷ slack held: a region using 5 per second and holding 1,000 refills about once every 200 seconds.

Cost

. During a , a region that runs out must refuse bonuses even though another region still has some. Reduce it with small refill blocks and a threshold-triggered refill.

Reject

"Count, check, then insert", at any : it is the write-skew shape again. Use a , or a table of tokens decremented transactionally in each region.

Read moreInvariant confluenceThe one escape: partition the slack

9A distributed store is configured with N = 3, R = 2, W = 2, sloppy quorums and last-write-wins timestamps. A colleague says its reads are strongly consistent. Prepare a review.Show the answer

Why overlap is not recency

  • guarantees overlap only.
  • Last-write-wins makes "latest" depend on clock skew.
  • With no write-back after a read, can fail.
  • A failed write can reappear later.
  • let the W acknowledgements come from nodes outside the key's usual replicas, so the overlap argument fails exactly during failures.

Ask for evidence

Run a list-append or register test with a history checker.

Recommend by use

A ledger needs a linearizable store, or a leader with consensus. A feed needs a stated staleness bound and session tokens. Counters can use CRDTs.

Also

LOCAL_QUORUM quietly redefines N as one data centre, and a sequence of calls at mixed levels gets the weakest pairing.

Read moreTunable consistencySloppy quorumsWhat is actually verifiable

10Support reports that a user sees their new display name, then the old one again on the next refresh. Diagnose and fix.Show the answer

Diagnose

A failure of monotonic reads and read-your-writes, not a bug. Most likely a recent infrastructure change (a load balancer, a CDN, retries to another region) now sends requests to replicas at different positions in the log, and the client carries no token.

Test

Log the replica ID and its log position for every request.

Fix

A token: the position, carried in a header or cookie, so the router picks a replica that has applied it or falls back to the primary. Or . Prefer the token: sticky routing turns a consistency guarantee into a load-balancing constraint that fights failover and autoscaling.

Make it owned

Name the owner of the token and add the guarantee to the SLO. Expected impact ≈ lag × request rate.

Read moreSession guaranteesGive the client a token

11Choose how to merge concurrent edits for three fields in a shared document store: the document title, the list of collaborators, and a view counter.Show the answer

Title

Last-write-wins is acceptable: any recent value will do, and its authority is the user who edited last. Warn that skewed clocks pick the winner, and a lost edit raises no error.

Collaborators

An observed-remove set (), where an add wins over a concurrent remove. It costs tags and , so plan the collector and monitor the oldest uncollected tombstone: one offline replica blocks collection.

View counter

A grow-only counter (), or a if decrements exist. Its metadata grows with the number of replicas, so avoid giving each client its own replica.

Convergence is not correctness

A rule such as "at most 10 collaborators" is an upper bound, not preserved by merging sets, so it needs escrow or coordination.

Read moreArbitrate: pick a winnerOR-SetG-CounterConvergence, not correctness

12Your CTO asks whether your two-region payments platform is 'CP or AP'. Answer, and say what you would measure.Show the answer

Reframe

The choice is made per operation, and only matters during a partition. Transfers and uniqueness checks, which are not invariant-confluent, must be CP: the minority side refuses them. Notes and counters can be AP, using CRDTs.

Measure what runs every day

The latency side of : p99 on a normal day for each operation, the distribution of replication lag, and how often the staleness bound is exceeded.

Also say

"CP" does not mean low : the system is unavailable only to the minority side, and only while partitioned. Identify , such as SMS or phone calls, that need external consistency or a token-based wait.

Read moreCAP, stated preciselyPACELC

13You are asked to prove that your database really provides snapshot isolation, and to find any write skew. What do you do?Show the answer

Stance

Treat the vendor's level as a hypothesis to test.

Method

Build a workload whose version order can be recovered from reads, such as appending to lists. Record the history, infer the with write–write, write–read and read–write edges, and search it for cycles. Do not search for a valid linearization: that is .

Classify

A cycle with exactly one read–write () edge is G-single, read skew, which snapshot isolation forbids. A cycle with two or more is G2, write skew, which snapshot isolation allows.

Report

Which cycle classes appear at which level, with the exact histories.

Caveat

A clean run does not prove absence; and unobserved operations cannot be checked this way.

Read moreWhat is actually verifiableThe anomalies are three shapes

14A design review lists ten different consistency settings across an application's screens. How do you decide what each screen actually needs?Show the answer

An ordered procedure

  1. Try to make the operation commute.
  2. Check whether the rule fits inside one row.
  3. Check whether the rule is invariant-confluent.
  4. If it is not, try to (escrow).
  5. If the application does read-modify-write itself, rule out Read Committed.
  6. If a rule over a set is checked and then written, rule out snapshot isolation.
  7. Ask whether any out-of-band channel is faster than replication.
  8. Only then choose the model, and write down the staleness bound.

The output

A table: screen, rule, the step that decided it, the choice, and who owns the token.

Ask of every cache and replica

How many observers, how far apart, and what does each believe?

Read moreWhat consistency forbidsApplying it screen by screen