Buffer survival under overload

How it behaves

One input on the horizontal axis, the others held fixed.

Worked examples, recomputed

Every number the mechanisms quote for this relationship is recomputed from it, so none of them disagrees with it.

• 600,000 events at 2,000 − 800 per second → 500 s
• a buffer ten times bigger → 5,000 s
• (2,000 − 800) × 7,200 s = 8,640,000 events
• 800 − 400 per second for an hour → 1.44 million
• a stuck consumer: 100 per second for an hour → 360,000
• 2,000 per second, commit every 5 s → up to 10,000 re-read
• 2 GB/s consumed, 3 s to free space → 6 GB of reserve
• 300 GB free ÷ 120 GB/day → 2.5 days
• 6,000 offered against 5,000 for 10 min → a new request waits 120 s
• after an hour → 720 s

Where it appears

Each mechanism states it in its own names. The general symbol each name stands for is shown with it.

• Data Movement

  How do records cross between two systems that have independent rates, independent availability and independent lifetimes, and what are you willing to lose when the assumption underneath the crossing breaks?

  $$\text{survival time}=\text{capacity}÷(\lambda -\mu )$$

  Read it in place →

• Change Data Capture & Incremental Processing

  What does it cost to know what changed, and is that cheaper than not needing to know?

  $$\text{days until the disk fills}=\frac{\text{free space}}{\text{log written per day}}$$

  free space = $C$ · log per day (reader stopped) = $\lambda -\mu$

  Read it in place →

• Measurement & Experimental Method

  What would have to be true for this number to mean what I think it means?

  $$\text{backlog}=(\lambda -\mu )t,\text{wait}=\text{backlog}/\mu$$

  Read it in place →