Interview questions

Because the producer and the consumer cannot be updated at the same moment, and the producer may no longer exist. How many parties can take part without negotiating matters more than speed.

Read moreWhat changes at a boundary

The time gap between writer and reader, and whether the reader can talk back to the writer. They give file formats (unbounded gap, no channel back), access protocols (short gap, a full channel back) and integration patterns (an indefinite gap, where the channel back must also remember where it got to).

Read moreTime gap and back-channel

The party with less power to refuse. Power, not efficiency, predicts which side does the work.

Read morePrinciple C: power picks the payerWho pays

The work between the bytes and the values: parsing characters, guessing types, decoding. The reader pays it on every read, forever.

Read moreThe parse tax

Splittability is whether a reader starting at an arbitrary byte can find the next record boundary, so that many workers can share one file. It is a property of the framing, not of the compression codec. One big gzip stream cannot be entered in the middle, but formats that cut the file into independent blocks stay splittable under any codec.

Read moreSplittability

A quoted field may contain a newline, so a worker that jumps into the middle and looks for the next newline may start inside a field. Newline-delimited JSON (NDJSON) fixes it with the rule "one record, one line".

Read moreText formatsNDJSON: framing is worth more than encoding

Reading 3 of 200 columns needs a directory of where each column is. The writer only knows all the offsets at the end, so that is where the directory goes. The file must therefore be finished before it can be read, and it cannot be streamed or tailed.

Read moreWhy there is a footer

• Nowhere: the reader infers it, paying the parse tax and the risk of guessing wrong.
• In the bytes: overhead in every file, which sets a minimum sensible file size.
• In a registry: tiny messages, but the registry is on the critical path, and losing it makes the data undecodable.

Read moreWhere does the agreed schema sit?

Character encoding; decimal versus float; null versus empty versus absent; timestamps and time zones; integer width; and collation (sorting rules). A "not null" check does not catch them, because the data is valid and simply means two things.

Read moreThe interop hazards

If the reply is lost, the sender sees the same silence whether the request failed before, during or after the action. This never goes away, so the receiver must be able to handle the same request twice (idempotency).

Read moreThe back-channel appearsWebhooks and callbacks

Mostly because engines have different capabilities, not because vendors are careless. A declarative interface lets the engine decide how to execute, so each engine exposes what it can do.

Read moreDialects, not vendor sloppiness

The cursor is row-oriented, stateful and synchronous, so a slow client holds the server's session. The three ways out are the cursor, columnar RPC, and unload (write files to shared storage and hand over a pointer). Only unload has a cost that does not depend on the client's speed.

Read moreWhat they force: the cursorResult egress: only three ways out

Any server can answer any request, so servers can be added and removed freely. The price is context re-sent on every request, and often several round trips per answer.

Read moreREST and GraphQLThe price is paid twice

It gains one round trip for any need, and loses a predictable cost per request. Mature services add depth limits, complexity budgets, cost estimation and allow-listed queries, which fix the set of query shapes again.

Read moreGraphQL: let the caller choose the shape

Protocol compatibility means clients can connect and exchange bytes, as with the Postgres wire protocol. Semantic compatibility means the same isolation, types, transactions and functions. The first is cheap; the second is not.

Read moreThe Postgres wire protocol

Duplicates, so the receiver must be idempotent; retry storms, so backoff, retry budgets and a dead-letter path; no guaranteed order; and the receiver's availability becoming part of the producer's pipeline.

Read moreWebhooks and callbacks

Cost = time both versions run × number of callers. With 400 integrations and a 12-month sunset, that is 4,800 integration-months.

Read moreVersioning and deprecation

Only when its position is total (any two positions can be compared) and monotonic (it only increases). A log offset qualifies. A timestamp high-watermark does not, if writes arrive out of order or share a millisecond.

Read moreConnector frameworksWhat is a position?

Reprocessability. ELT keeps the raw data, so after a bug you recompute inside the destination. ETL shields the destination from schema drift at the source, but forces you to extract again. The industry moved because storage became cheap and compute elastic, an economic fact that could flip back.

Read moreELT versus ETLWhy the industry moved

How selective the part that can be pushed down to the source is. With 10⁹ rows of 100 bytes and a filter keeping 0.1%, pushing the filter down moves 100 MB, and not pushing it moves 100 GB: a 1,000× difference from a mismatch in capability.

Read moreFederation and query pass-through

Under concurrent inserts or deletes, rows shift between pages, so a client sees a row twice or never sees it, with no error. Keyset pagination anchors to a value instead, but needs a unique, stable, ordered key.

Read morePagination and sync tokensKeyset pagination

Every other failure produces a signal, such as an error or a wrong value. A missing file produces nothing, so it can only be noticed by expecting it on a schedule.

Read moreFile drop and partner exchange