The interop hazards

The six hazards

The defects that really happen at this boundary are almost never format-level. They are agreement-level, and the set is small and stable:

1. Character encoding
2. Decimal versus binary floating point
3. Null versus empty versus absent
4. Timestamp meaning (instant versus local time, and the writer's zone)
5. Integer width and overflow
6. String collation (the sort order of text)

What unites them: both sides parse successfully and disagree silently. There is no error to catch and no exception to log. The file is valid, the read succeeds, and the number is wrong. These are exactly the assumptions a format cannot verify, and the answer to "who pays?" is always the consumer, downstream, late. This is the top of the real defect list at this boundary, and it gets less attention than the exciting topics above.

Here are small examples of each one.

1. Character encoding. The byte 0xE9 is é in Latin-1 but is invalid on its own in UTF-8 (where é is the two bytes 0xC3 0xA9). A city name São Paulo written in one encoding and read in another may become São Paulo. Nothing fails.

2. Decimal versus binary floating point. Binary floating point cannot store most decimal fractions exactly:

>>> 0.1 + 0.2
0.30000000000000004
>>> sum([0.1] * 10)              # ten prices of 0.10
0.9999999999999999

If the writer used exact decimal (cents as integers, or a decimal type) and the reader uses float, the totals differ by tiny amounts. This is exactly the few-cents difference finance saw in Priya's story. (The exact digits depend on the language and the order of additions; the point is that they are not exact.)

3. Null, empty, absent. These are three different states:

{"discount": null}     // known to be "no value"
{"discount": ""}       // an empty string
{}                     // the field was never sent

In CSV, a,,c cannot say whether the middle field is null or an empty string. Two readers can pick different meanings, and an average that treats null as 0 may differ from one that skips it.

4. Timestamps. 2026-03-08 02:30:00 has no zone. Is it UTC? The writer's local time? In many time zones, that local time did not exist on that date (daylight-saving jump) or happened twice. An instant (one point on the global timeline) and a local time (a wall-clock reading) are different things. Without the writer's zone, the reader can only guess.

5. Integer width. A 32-bit signed integer holds at most 2,147,483,647. The next ID, 2,147,483,648, wraps to a negative number in a reader that uses 32 bits, while the writer used 64.

6. Collation. Sorting ["apple", "Zebra", "banana"] by raw bytes gives Zebra, apple, banana (uppercase letters come before lowercase in ASCII). A case-insensitive collation gives apple, banana, Zebra. If one system sorts and another merges or joins assuming a different order, the results are wrong without any error.

(Note on scope: a constraint check, such as "this column must not be null", finds violated rules. These six hazards are different: the format is valid and every rule is satisfied, but the two sides give the data different meanings.)

What file formats forbid

• You cannot have zero parse cost and zero prior agreement. Someone interprets; the only question is once or always.
• You cannot have random access into a file that has no ending.
• You cannot have a self-describing set of files. Describing a collection always needs a second, higher-level format.
• You cannot make any of these changes atomically once files exist, because the files are the deployed artifact and you cannot redeploy them.

The escapes and their prices