Skip to content
The Data EngineeringThe Data EngineeringBeta
About
Main content

Sensors: polling versus push

Why do waiting sensors deadlock a pool, and what does polling or push cost?

A sensor is a task whose only job is to wait until a condition becomes true, usually by (checking again and again at a fixed interval). This is the question of push versus pull in its purest form, and the derivation is in the . It is the same law that governs disk queue depth and network buffers, now applied to a system that moves no data.

Sensors hold slots

A polling sensor holds a resource for the whole time of its wait. If λ sensors start per hour and each waits W hours, then sensors occupy slots at once.

This is not just a performance problem. It is a deadlock: sensors fill every slot while waiting for tasks that need a slot to run, so none can be scheduled. The system is fully used and making no progress. It is a resource-exhaustion cascade caused by the control plane itself.

Waiting must not hold the slot it waits forpool 100 · 300 sensors × 4 h

The fix: separate waiting from occupying

Every mature found the same fix independently: separate waiting from occupying. The sensor registers its condition with a small shared poller and releases its slot. It resumes only when the condition fires. Names vary: deferrable operators, async sensors, and so on.

The agreement among independent products is evidence of a general principle, not a product feature:

It is the same restructuring as one blocking thread per connection versus an (a single thread that serves many connections and runs code only when something happens), three levels of abstraction apart, for the same reason.

Polling cost

Apart from deadlock, polling cost is an ordinary batch calculation:

  • Mean detection latency ≈ half the poll interval (the event arrives at a random point between two polls).
  • .

Example. On an object store, a LIST request (list the objects under a prefix) is billed per request and is priced above a GET (read an object). A one-minute poll over a four-hour wait makes 4 × 60 = 240 LIST calls per sensor per run. The number is invisible for one sensor and material across a fleet: 300 sensors × 240 = 72,000 LIST calls per night. This is per-request cloud pricing reaching into the control plane.

Push

Push It is cheaper and faster. But it requires the producer to know the consumer exists, which is coupling. It also requires delivery guarantees that the producer probably does not have:

  • A lost notification is indistinguishable from "not ready yet", and it fails silently forever.
  • A duplicated notification triggers the run twice.

So push needs triggering (the same trigger twice starts one run) and, in practice, a slow polling fallback underneath. Belt and braces: the belt (push) gives latency, and the braces (slow polling) give .

Waiting must not consume the resource being waited for; polling cost is Little's Law in the control plane, and push buys latency at the price of a coupling and a lost-message failure mode.