Priority routing and multiplexing

How do you serve high-priority work first without starving the rest, and what does sharing one channel cost?

Priority routing

Not all records are equal. Priority routing serves the high-priority work first.

Separate queues per priority with weighted consumption is the practical answer. For example, take 10 records from high, 3 from medium and 1 from low in each round. There is no priority comparison and no reordering. The weights prevent starvation naturally, and each priority can be scaled independently. This is nearly always what you want.

A single priority queue gives true priority ordering. It costs O(log n) per insert and unbounded state (it must hold everything to know the maximum). It also fights every other mechanism here: it cannot be partitioned without breaking global priority, it cannot stream, and it inverts batching.

Priority at admission rather than in transit is usually the better choice. (Admission control is from Flow control.) Instead of ordering a backlog, reserve capacity for high-priority work and shed low-priority work under load. You move the decision from "which to do next" to "which to accept". That is cheaper and has better failure behavior.

Starvation

The defining problem is starvation: under sustained high-priority load, low-priority work never runs. Fixes:

Aging. A record's priority rises with its waiting time. This converts starvation into long but bounded latency, which is at least visible.
Reserved capacity per class (bulkheading again).
Weighted fair queueing. Each class gets a guaranteed share, and unused share is redistributed to the others. This is the best general answer.

The one that hurts

Priority interacts badly with partitioning. With P priorities and K partitions you have P·K queues and need a consumption policy across all of them. Worse: if a high-priority and a low-priority record share a key and you route them to different queues, you have broken per-key order in order to honor priority. Priority and per-key ordering are in direct conflict. For a given key you can have one or the other, not both. Most systems that claim both are quietly not delivering one.

The observation worth carrying

Priority matters only when you are saturated. With spare capacity, everything is served quickly and priority does nothing. So priority routing is really an overload mechanism. Design it together with load shedding and admission control, not as a routing feature. If you need priority routing routinely, you are chronically under-provisioned, and that is the real finding.

Multiplexing and demultiplexing

Multiplexing means carrying many logical streams over one physical channel. Demultiplexing means separating them again at the other end. Examples: HTTP/2 streams over one TCP connection, gRPC channels, one Kafka connection carrying many partitions, many tenants' records sharing one topic.

Why: connection setup is expensive (the TCP handshake plus the TLS handshake), connections are a limited resource, and the per-connection overhead multiplied by thousands of streams is too much. Multiplexing amortizes all of it.

The cost that defines it is shared fate. One physical channel means one failure domain, one flow-control domain (unless there is per-stream flow control; this is exactly why HTTP/2 has both connection-level and stream-level windows) and one risk of head-of-line blocking.

The canonical example

HTTP/2 multiplexes streams over TCP. This solves application-layer head-of-line blocking: a slow response no longer blocks the others. But TCP guarantees in-order delivery of bytes. So one lost packet stalls every multiplexed stream until it is retransmitted. HTTP/2 removed head-of-line blocking at layer 7 (the application) and left it at layer 4 (the transport). QUIC exists to fix this: independent streams over UDP, with loss recovery per stream.

The general lesson reaches well beyond networking: multiplexing moves head-of-line blocking down one layer instead of removing it. If the layer below preserves order across your logical streams, you have not isolated them. You have only made the coupling less visible. Whenever you multiplex, ask what the layer underneath guarantees, because the coupling you did not intend will live there.

Demultiplexing needs an identifier

The framing must carry a stream identifier (an HTTP/2 stream ID, a gRPC stream, a tenant ID header), and the receiver needs a dispatch table. Where the identifier lives is the same header-versus-body decision as in content-based routing: put it in the frame, not in the payload, or the demultiplexer must deserialize every record.

In data engineering: shared topic or topic per tenant

You can use one topic with a tenant key, or one topic per tenant. A shared topic costs you isolation: one tenant's burst affects everyone, and one poison record can stall a shared partition. It buys you simple metadata: 10,000 topics is an operational problem, while 10,000 keys is not. The standard resolution is tiered: shared topics for the long tail of small tenants, dedicated topics for the large or noisy tenants. That is bulkheading applied selectively: isolate where it is worth paying for.