Schema-on-write versus schema-on-read

Why is validation work only ever moved, and why is binding a per-field decision?

CaseConstructed case · Sara at Parcelo

Two dashboards, two numbers for the same week

To avoid "blocking the producer", Parcelo's platform team lands raw events into a raw_events table as plain JSON text, with no checking. Each team then reads it in its own way. On Monday, two dashboards show different numbers for "parcels delivered last week": 41,200 on one, 40,870 on the other. Both teams say their query is correct. There is no document that says what a "delivered" event looks like, and the two queries parse it differently.

On the same Monday a nightly job fails to join two tables. Its automatically detected schema now says postal_code is text. Last week it was an integer. A merchant had started sending postcodes like "SW1A 1AA".

A third request arrives. A big merchant wants to send its own custom fields with each shipment (a gift message, a warehouse bin number). Ravi cannot add a column for every merchant. These three problems have one origin: where the structure is bound.

How it ends →

Validation work exists between production and consumption. It cannot be deleted. It can only be placed. That is the whole derivation. Everything else is a choice of where to put it.

Write-time placement (schema-on-write): data is checked when it is written.

There is one enforcement point, and its guarantee is inherited by all N readers.
The producer is blocked until its data conforms. Ingestion fails on a violation.
Structural change must go through an authority.

Read-time placement (schema-on-read): data is stored as it arrives and interpreted when it is read.

The producer is not blocked. Everything is kept, whatever its shape.
But validation runs per reader, per read. That is a CPU cost, but it is the small cost.
The big cost: each reader now has a private, unversioned, undocumented schema.

The characteristic failure of schema-on-read is not an unparseable file. It is two dashboards that disagree, and no artifact to compare. (This is Sara's Monday.)

The common description is wrong

There is no such thing as schema-on-read. There is:

declared once, centrally, and
inferred repeatedly, privately.

The second is not the absence of a schema. It is N schemas with no owner. The work was multiplied and hidden, not removed.

The late-bound decision lands on the party with the least information about intent. The producer knew what the field meant. The consumer is guessing. Every late-binding mechanism (schema-on-read, inference, coercion, permissive parsing, reader-side defaults) moves an interpretation from someone who knew to someone who does not. The gap is filled with a guess that is indistinguishable from data. The question "when does binding happen?" gives timing. This law says who ends up deciding, and that is what makes the resulting error silent instead of loud.

It is a per-field decision, not a system-wide choice

The other half of the correction: this is not a yes/no choice for a whole system. It is a decision per field. The answer that real systems converge on is:

The core is the small set of fields that everyone needs and that must be right: keys, timestamps, tenant, event type, version. These are declared and checked at write time. The payload is the part that varies, and it is bound late. Both purist positions ("everything must be declared" and "never declare anything") are folklore, kept alive mostly by people who sell one of them.

Validation is conserved: write-time buys one schema and blocks the producer. Read-time buys producer freedom and yields N private schemas.