The sensor deadlock

Break it on purposeNeeds Python

Problem

A sensor is a task that waits for a condition, often the completion of another task. If it waits while holding a worker slot, it consumes capacity doing nothing; and if enough sensors wait at once, they hold every slot, the tasks they are waiting for cannot start, and the pool deadlocks until the sensors time out. Little's Law says how many slots waiting costs: the number of sensors in flight is their arrival rate times how long each waits.

Simulate a worker pool of 10 slots with threads. Start 20 "sensor" tasks that each hold a slot while waiting (up to 5 s) for a processing task, and 20 "processing" tasks that must acquire a slot to run. Then change the sensor to release the slot while waiting (acquire only to check, sleep outside the semaphore). Compute L = λW first and check that it predicts the problem.

The question. How long until the first processing task runs when sensors hold their slots, and when they release them while waiting?

Predict

With 20 sensors holding all 10 slots and a 5 s timeout, when does the first processing task start?

At onceAfter about 5 s, the timeoutAfter about 10 sNever

Run it

1Install uv once. It fetches Python and the lab's packages for you.
Terminal
```
curl -LsSf https://astral.sh/uv/install.sh | sh
```

2Get the script. It is the whole lab: read it, then download it.

the-sensor-deadlock.py58 linesDownload

View the code

the-sensor-deadlock.py

# /// script
# requires-python = ">=3.10"
# ///
# Copyright (c) 2026 Aayush Sharma · SPDX-License-Identifier: MIT
# https://thedataengineering.com/lab-code/LICENSE.txt
"""The sensor deadlock.

A worker pool of 10 slots (a semaphore). 20 "sensor" tasks each hold a slot
while they wait for a processing task to finish; 20 "processing" tasks need a
slot to run. All sensors start first. Then the same with sensors that release
their slot while they wait (acquire only to check, sleep outside the pool).
"""
import argparse, json, threading, time

p = argparse.ArgumentParser(description=__doc__.splitlines()[0])
p.add_argument("--slots", type=int, default=10, help="Pool slots.")
p.add_argument("--sensors", type=int, default=20, help="Sensor tasks.")
p.add_argument("--tasks", type=int, default=20, help="Processing tasks.")
p.add_argument("--timeout", type=float, default=5, help="Sensor timeout (s).")
p.add_argument("--json", help="Also write the results to this JSON file.")
args = p.parse_args()

def run(release_while_waiting):
    pool = threading.Semaphore(args.slots); done = threading.Event(); first = []; t0 = time.perf_counter()
    def processing(i):
        with pool:                                                     # needs a slot
            if not first: first.append(time.perf_counter() - t0)
            time.sleep(0.1)
        done.set()
    def sensor(i):
        if release_while_waiting:
            while True:
                with pool:                                             # hold a slot only to check
                    if done.is_set(): return
                time.sleep(0.05)                                       # wait outside the pool
                if time.perf_counter() - t0 > args.timeout: return
        else:
            with pool:                                                 # holds a slot while waiting
                done.wait(timeout=args.timeout)                        # waits for a processing task
    ts = [threading.Thread(target=sensor, args=(i,)) for i in range(args.sensors)]
    for t in ts: t.start()
    time.sleep(0.05)                                                   # all sensors first
    ps = [threading.Thread(target=processing, args=(i,)) for i in range(args.tasks)]
    for t in ps: t.start()
    for t in ts + ps: t.join()
    return {"sensor": "release the slot while waiting" if release_while_waiting else "hold the slot while waiting",
            "first_task_s": first[0], "total_s": time.perf_counter() - t0}

rows = [run(False), run(True)]
# Little's Law: sensors arrive together and each would hold a slot for W = timeout, so L = sensors in flight
summary = {"slots": args.slots, "sensor_demand": args.sensors, "lambda_w": args.sensors}
print(f"L = λW: {args.sensors} sensors each waiting up to {args.timeout:g} s want {args.sensors} slots at once; the pool has {args.slots}")
for r in rows: print(f"{r['sensor']:<32} first processing task started after {r['first_task_s']:.2f} s; all done after {r['total_s']:.2f} s")

if args.json:
    with open(args.json, "w") as f:
        json.dump({"summary": summary, "tables": {"runs": rows}}, f)

3Run it in the folder you saved it to, with the defaults first. Then change a knob from the table below and run it again.
Terminal
```
uv run the-sensor-deadlock.py
uv run the-sensor-deadlock.py --help     # every knob
```

Knobs
Knob · default	What it changes
--slots10	Pool slots.
--sensors20	Sensor tasks.
--tasks20	Processing tasks.
--timeout5	Sensor timeout (s).

Our run

Two kinds of sensor

Two kinds of sensor
Sensor	First processing task	All done
hold the slot while waiting	10 s	10.2 s
release the slot while waiting	55.4 ms	266 ms

Reference runApple M2 · 8 cores · 8 GB

Output · uv run the-sensor-deadlock.py

L = λW: 20 sensors each waiting up to 5 s want 20 slots at once; the pool has 10
hold the slot while waiting      first processing task started after 10.01 s; all done after 10.22 s
release the slot while waiting   first processing task started after 0.06 s; all done after 0.27 s

macOS 26.6.2 · Python 3.13.3 · run 2026-10-20

What we found

L = λW predicted the problem: 20 sensors waiting at once want 20 slots from a pool of 10. Holding their slots, they delayed the first processing task by 10 s, twice the timeout, not once: when the first ten sensors timed out, the slots went to the ten sensors queued behind them, ahead of the processing tasks. Releasing the slot while waiting, the first task started after 55.4 ms and everything finished in 266 ms.

Read it

A timeout does not break this deadlock, it only rotates it: as long as more waiters queue for slots, each timeout hands the slot to the next waiter. The fix is structural: waiting must not hold a slot (rescheduling or deferrable sensors), or waiters must have a pool of their own.