Interview questions

Probably not: the benchmark is most likely issuing one request at a time (queue depth 1). At about 80 µs per request, one request at a time gives 1 / 80 µs = 12,500 IOPS. Reaching 1 million needs about 80 requests in flight at once, by Little's Law. The drive is fast only when it is given parallel work.

Read moreNVMe

Every request to an object store pays a fixed minimum latency of 20–100 ms and a per-request fee, however few bytes it returns. A thousand small reads is about 50 seconds of waiting if done one after another. Compact small files into large ones, and read large byte ranges in parallel, so the fixed cost is spread over many bytes.

Read moreObject storageFixed cost per operation

Flash cannot overwrite in place; it erases whole blocks. Small random updates mix data of different ages inside each block, so the drive's garbage collector must copy the still-live pages elsewhere before erasing, and the drive writes much more than the host asked (write amplification). LSM-trees write large files whose contents all age together, so the drive's amplification is close to 1. The cost does not vanish: it moves into the LSM's own compaction.

Read moreWrite amplificationWhy LSM-trees exist

The drive does not know a file was deleted; only the file system does. Until TRIM tells the drive which blocks are free, it treats them as live data, so it stays effectively full and keeps copying them during garbage collection. The slowdown comes from missing information, not from the physics of flash.

Read moreNAND flash (SSDs)

S3 guarantees consistency per object only, and it has no real folders. Renaming a "folder" is N copies and N deletes, and it can stop halfway. Table formats such as Iceberg and Delta never rename: they write new files, then commit by swapping one pointer to a new metadata file in a single atomic step, and make every other step safe to repeat.

Read moreObject storage

The delay is the provider's batch-scheduling policy, not the physics of tape. Requests are queued and served in batches so the provider can keep its tape drives and robots fully used; you are paying a low price in exchange for waiting.

Read moreTape and cold tiers

Assume the cost was moved, not removed, and ask where it went: extra space, slower reads, background compaction, more memory, or work pushed onto another component or team. Every storage design pays for its writes somewhere; the useful question is which of those you would rather pay.

Read moreCosts are conserved, never removed

No. The advantage is 100–1,000× on a hard disk, 2–10× on an SSD, 5–10× in DRAM, and nothing on object storage. What "sequential" really buys is large requests (amortizing the fixed cost) and predictable ones (which can be fetched ahead).

Read moreSequential versus random access

Memory moves in 64-byte cache lines. In a row layout, a query that needs one column out of twenty still drags the other columns through every cache line, about 20× more bytes than it uses. In a column layout every byte fetched is useful, access is sequential so the hardware prefetcher keeps up, and values of one type next to each other compress much better.

Read moreCache misses dominate scansThe case for columnar storage

False sharing. The counters sit in the same 64-byte cache line, and a cache line is owned by one core at a time, so every update forces the line to move between cores. Pad each counter onto its own cache line, or keep thread-local counters and combine them at the end.

Read moreFalse sharing

Because the branch becomes predictable. On sorted data the filter's condition is false for a long run and then true for a long run, so the CPU's branch predictor is almost always right. On random data it guesses wrong about half the time, and each wrong guess costs 15–20 cycles of discarded work.

Read moreBranch prediction

Not with acceptable latency. A round trip between continents takes 70–250 ms, and consensus needs one or two round trips per commit, so every write would take hundreds of milliseconds. Keep each quorum within one region, across its availability zones (1–2 ms apart), and replicate between regions asynchronously.

Read moreThe latency tiers

To fill a link, the sender must keep bandwidth × round-trip time of data in flight, the bandwidth-delay product. At 10 Gb/s and 100 ms that is about 125 MB, while default TCP windows allow far less, so the sender spends most of its time waiting for acknowledgements. Use several parallel streams or larger TCP buffers.

Read moreLatency and bandwidth are independent

A shuffle is all-to-all traffic: every machine sends to every other. Data-centre networks are priced and built assuming most traffic stays local, with fewer links higher up (oversubscription), so all-to-all traffic hits the thinnest part of the network. And because many senders reply to one receiver at the same moment, shuffles also trigger incast.

Read moreBisection bandwidthIncast

Incast. When many workers reply to one receiver at the same instant, their packets overflow the switch port's small buffer and some are dropped. Small flows cannot detect the loss quickly and recover only after TCP's retransmission timeout, 200 ms or more, so throughput collapses although average utilization is low. Add jitter to replies, limit fan-out, or use ECN-based congestion control such as DCTCP.

Read moreIncast

Into the service, not the distance: load balancers, TLS, authentication and metadata lookups each add a step. That is why moving your compute closer to S3 does not reduce the minimum latency; only fewer, larger requests do.

Read moreThe latency tiers

Because waiting time grows sharply long before 100%. In a simple queue, response time ≈ S / (1 − ρ), where S is the time to serve one request and ρ is utilization. At ρ = 0.8 the average request already takes 5 times its service time, the tail is longer still, and any burst pushes the queue toward the wall.

Read moreUtilization and the queueing wallWhy 80% has already used up the budget

Coordination between servers can grow with the square of their number, since every pair may need to agree. The Universal Scalability Law models this with a coherence term: past a certain size, each added server costs more in coordination than it contributes in work, and throughput falls. Reduce coordination, for example by sharding or partitioning so that servers rarely need each other.

Read moreUniversal Scalability Law

A request is as slow as its slowest backend. If each backend is slow 1% of the time, a request touching 100 of them meets at least one slow backend with probability 1 − 0.99¹⁰⁰ ≈ 63%. So the user-facing p99 is set by roughly the backends' p99.99. Reduce fan-out, or send a duplicate (hedged) request when one is slow.

Read moreTail at scale

Part of the job does not parallelize: planning, the final commit, and waiting for the slowest tasks. By Amdahl's Law, a serial fraction s limits the speedup to 1 / s however many machines you add. What the bigger cluster does buy is the ability to process about 10 times more data in the same time (Gustafson's Law).

Read moreAmdahl vs. Gustafson

By Little's Law (concurrency = throughput × time in the system), throughput is at most 50 / 0.1 s = 500 queries per second. Beyond that, requests wait for a connection, and their latency grows.

Read moreLittle's Law

Because they feed the queue that caused the timeouts. Timeouts trigger retries, retries raise the arrival rate, a higher arrival rate raises utilization, and higher utilization makes every request slower, so more of them time out. Break the loop with retry budgets, circuit breakers and load shedding.

Read moreWhy 80% has already used up the budget

To free a segment it must first move the still-live data out. If a fraction u of each cleaned segment is still live, it moves u / (1 − u) bytes per byte freed: 1 at u = 0.5, 9 at 0.9, 49 at 0.98. Free space is write amplification bought in advance.

Read moreWhat the collector costs

Which conserved quantity did it economize, and where did the cost go? Costs are moved, not removed; a claim with no answer to that is marketing. The ratios behind it are diverging (capacity grows faster than bandwidth, bandwidth faster than latency), which pushes every design toward moving less data and computing where the data already is.

Read moreWhy these limits last