Which of N destinations gets this record?

Which four properties does every routing choice trade, and why is routing the hardest decision to reverse?

CaseConstructed case · Ravi at Cartwheel

One partition hours behind, and time running backwards

Cartwheel now publishes every order event to a Kafka topic named orders. The topic has 12 partitions, and Ravi chose seller_id as the partition key, because "everything downstream is about sellers": seller dashboards, payouts, seller-level counts.

On Black Friday the order rate rises from the normal 300 per second to 2,000 per second. Lena's dashboard shows a strange picture. Eleven of the twelve partitions have a lag of nearly zero. One partition is hours behind, and the gap grows every minute. Sana adds six more consumers to the consumer group. Nothing changes. The new consumers sit idle.

The next morning there is a second problem. For a few orders, the OrderShipped event was processed before the OrderPlaced event. The warehouse service tried to ship an order it had never heard of. The orders involved were all ones that had failed once, been sent to a dead-letter queue and been replayed.

Omar asks: "We have 12 partitions and 12 consumers. Why is one of them doing all the work? And why is time running backwards?" The answers are in this article.

How it ends →

Transport patterns asked how a record crosses the gap between two systems. Flow control asked what happens when the rates of the two sides differ. Routing asks the question that sits between them and quietly determines both:

A destination here can be a partition of a topic, a consumer, a database shard or a cache node. Every mechanism in this article is an answer to that question, and every answer is a purchase against the same four properties. You cannot maximize all four. This table is the whole article in small form:

Property	What it means	Bought by	Paid in
Balance	the load is uniform across destinations	randomness, fine granularity	locality, order
Locality	related records land together	keyed or range assignment	balance (skew)
Order	the sequence inside a scope is preserved	one destination per key	parallelism
Stability	the assignment survives a change in the set of destinations	consistent or rendezvous hashing	uniformity, simplicity

Balance and locality are directly opposed. Perfect balance means related records are scattered. Perfect locality means the load follows the shape of your data, and your data is not uniform. That is the central tension of this article.

Two more facts decide how much this matters.

Order costs parallelism, and here it becomes a number. Earlier we saw that keeping records in order limits how much work can run in parallel. In a partitioned system this is exact: your partition count is both your parallelism ceiling and your ordering scope. One integer has two meanings, and they pull in opposite directions. More partitions allow more parallel consumers, but each partition is a smaller scope of order.

Routing decisions are the hardest to reverse. A batch size is a configuration change. A partitioning key is a data migration, a reprocessing job and a period in which old and new layouts must both work. If you get it wrong, you live with it for years. So it deserves more of your design time than almost anything else in the pipeline.