Skip to content
The Data EngineeringThe Data EngineeringBeta
About
Main content

Point-to-point versus publish/subscribe

Does each record go to one consumer or to every interested one, and what does each way of building fan-out cost?

This is the addressing dimension. It is fully independent of versus log: you can build either over either. The only question is: does each record go to one logical , or to all interested consumers?

  • Point-to-point: and consumer share a named channel. There is one recipient per message. Adding a second consumer means splitting the load, not duplicating it.
  • Publish/subscribe (pub/sub): the producer names a , not a recipient. Subscribers register for the topic. Each gets a copy. The producer does not know who or how many.

The value of is entirely in that last sentence: the producer's code does not change when a consumer is added. It is the extensibility property, and it is why event-driven architectures are pub/sub. When Cartwheel adds a fraud-scoring service next year, Orders will not need any change.

The implementation underneath matters more than the pattern

There are two ways to build (one record, many subscribers), and the difference is a real architectural decision.

Copy-per-subscriber (Amazon SNS to SQS queues, a RabbitMQ exchange to N queues, EventBridge to N targets):

  • N times the write work and N times the storage.
  • Subscribers are fully independent: their own progress, own retries, own , own poison messages, own .
  • A slow subscriber's backlog is that subscriber's problem alone.
  • Adding a subscriber costs a new queue. Removing one costs nothing.

Shared log with N cursors (Kafka (a set of consumers that share the work of one topic), Pulsar subscriptions, Kinesis consumers):

  • Write once, store once. This is cheap at high fan-out.
  • Subscribers are coupled through the shared substrate: they share the retention window, the count and the capacity.
  • A subscriber that falls behind the retention window does not just slow down. It loses data. That is a different and worse failure than a growing backlog.
  • Adding a subscriber costs nothing at write time, but it uses read bandwidth and page-cache on the brokers.

Choose copy-per-subscriber when subscribers have very different reliability profiles or SLAs and you want to limit the of each. Choose a shared log when the volume is high enough that N times the storage is real money, and you want .

The coupling in the shared-log case is subtle, so we state it plainly: When someone shortens retention from 7 days to 3 to save money, they have silently reduced every consumer's maximum tolerable outage to 3 days. That is a dependency between teams with no interface to review it.