Skip to content
About
Main content

What changes at a boundary

What is different once producer and consumer cannot be updated together?

Up to now, think of systems where you control both ends. The engine that writes a data page also reads it back. The program that writes a log file also replays it. In that situation you choose a data representation on physical merit: how it fits in the CPU cache, how many disk seeks it needs, how well it compresses, how many dollars it costs to scan a terabyte. You can do that because you can change the reader and the writer in the same deployment.

At a boundary, that assumption dies, and the deciding rule dies with it. This mechanism is defined by one structural fact:

What appears is a coordination constraint, not a physical one. Efficiency stops being the deciding vote. Look at CSV (comma-separated text). On every physical measure (bytes per second, CPU cost, compression) it is hard to defend. It is still one of the most widely read data formats in existence (a common claim; it is hard to measure exactly). The reason is that the thing being optimized is no longer bytes per second. It is the number of parties who can take part without negotiating.

So the governing question shifts:

Inside your own systemAt a boundary
What is the fastest representation?What can two parties agree on when they cannot talk to each other?
Change the reader and writer together.They change at different times, or one is gone.
Decided by physics (, seeks, compression).Decided by agreement and by how many parties already agreed.

And because agreement compounds (each extra reader makes the format more valuable to the next writer), this is the first topic across the platform where a mechanism's fitness is set by an economic process outside the system's own physics. Formats do not win on quality. They win on installed base (the number of existing users), and then they are very hard to kill.