Identity by position

How does a reader map bytes to fields, and why is position-based identity silently fatal?

The question: bytes in, fields out

A reader has bytes on disk and a schema in its head. It must decide which bytes correspond to which field. The rule that makes this decision is the identity rule. The entire set of legal operations on a schema follows from the identity rule you choose.

There are three common rules. In each one, "identity" means: what do I use to say "this is the same field as before"?

Identity rule	The field is identified by	Typical formats
Position	its ordinal (first, second, third column)	headerless CSV, some binary row formats
Name	a string	JSON, CSV with header, Avro (by field name), many databases
ID	an integer assigned when the column is created	table formats such as Iceberg, Protocol Buffers field numbers

Identity is the ordinal. It has zero lookup cost, and it is the only option for formats that carry no description of themselves (nothing in the file says what the columns are).

What it does to changes:

Rename is free and meaningless (the name is not used anywhere).
Insert in the middle, reorder and drop are catastrophic. And, the important point, they are silent: neighbouring columns are often of compatible types, so you get wrong values instead of an error.

That is why headerless delimited text is the worst-behaved interface in common use. Ravi's CSV is an example:

old file:   S1001,12.5,1999      <- shipment_id, volume_kg, price_cents
new file:   S1001,3.2,12.5,1999  <- shipment_id, weight_kg, volume_kg, price_cents

partner loader (still expects 3 columns by position):
  shipment_id = S1001
  volume_kg   = 3.2      <- actually weight
  price_cents = 12.5     <- actually volume  (a number, so it parses fine)
  (the fourth value is ignored or causes a quiet truncation)