Law

Alert precision is a base-rate problem

Page on symptoms the consumer experiences, not on every cause that might produce them.

precision = \frac{r \cdot recall}{r \cdot recall + ( 1 - r ) \cdot FPR}

$r$: Base rate; how often the real thing happens
$FPR$: False-positive rate; how often the detector fires on healthy checks

Recurring principle: Absence is not evidence

How it behaves

One input on the horizontal axis, the others held fixed.

1% false positives0.1%

Data table

Share of alerts that are real, by incident base rate (recall 100%)
Base rate of real incidents	series	Precision
0	1% false positives	0.1%
0	1% false positives	0.112%
0	1% false positives	0.126%
0	1% false positives	0.141%
0	1% false positives	0.158%
0	1% false positives	0.178%
0	1% false positives	0.199%
0	1% false positives	0.223%
0	1% false positives	0.251%
0	1% false positives	0.281%
0	1% false positives	0.315%
0	1% false positives	0.354%
0	1% false positives	0.397%
0	1% false positives	0.445%
0	1% false positives	0.499%
0	1% false positives	0.559%
0	1% false positives	0.627%
0	1% false positives	0.703%
0	1% false positives	0.788%
0	1% false positives	0.883%
0	1% false positives	0.99%
0	1% false positives	1.11%
0	1% false positives	1.24%
0	1% false positives	1.39%
0	1% false positives	1.56%
0	1% false positives	1.75%
0	1% false positives	1.96%
0	1% false positives	2.19%
0	1% false positives	2.45%
0	1% false positives	2.74%
0	1% false positives	3.07%
0	1% false positives	3.43%
0	1% false positives	3.83%
0	1% false positives	4.28%
0.001	1% false positives	4.77%
0.001	1% false positives	5.33%
0.001	1% false positives	5.94%
0.001	1% false positives	6.62%
0.001	1% false positives	7.36%
0.001	1% false positives	8.19%
0.001	1% false positives	9.1%
0.001	1% false positives	10.1%
0.001	1% false positives	11.2%
0.001	1% false positives	12.4%
0.002	1% false positives	13.7%
0.002	1% false positives	15.1%
0.002	1% false positives	16.7%
0.002	1% false positives	18.3%
0.003	1% false positives	20.1%
0.003	1% false positives	22%
0.003	1% false positives	24.1%
0.004	1% false positives	26.3%
0.004	1% false positives	28.6%
0.004	1% false positives	31%
0.005	1% false positives	33.5%
0.006	1% false positives	36.1%
0.006	1% false positives	38.8%
0.007	1% false positives	41.6%
0.008	1% false positives	44.5%
0.009	1% false positives	47.3%
0.01	1% false positives	50.3%
0.011	1% false positives	53.2%
0.013	1% false positives	56%
0.014	1% false positives	58.9%
0.016	1% false positives	61.7%
0.018	1% false positives	64.4%
0.02	1% false positives	67.1%
0.022	1% false positives	69.6%
0.025	1% false positives	72%
0.028	1% false positives	74.4%
0.032	1% false positives	76.6%
0.036	1% false positives	78.6%
0.04	1% false positives	80.6%
0.045	1% false positives	82.4%
0.05	1% false positives	84.1%
0.056	1% false positives	85.6%
0.063	1% false positives	87.1%
0.071	1% false positives	88.4%
0.079	1% false positives	89.6%
0.089	1% false positives	90.7%
0.1	1% false positives	91.7%
0	0.1%	0.99%
0	0.1%	1.11%
0	0.1%	1.24%
0	0.1%	1.39%
0	0.1%	1.56%
0	0.1%	1.75%
0	0.1%	1.96%
0	0.1%	2.19%
0	0.1%	2.45%
0	0.1%	2.74%
0	0.1%	3.07%
0	0.1%	3.43%
0	0.1%	3.83%
0	0.1%	4.28%
0	0.1%	4.77%
0	0.1%	5.32%
0	0.1%	5.94%
0	0.1%	6.61%
0	0.1%	7.36%
0	0.1%	8.18%
0	0.1%	9.09%
0	0.1%	10.1%
0	0.1%	11.2%
0	0.1%	12.4%
0	0.1%	13.7%
0	0.1%	15.1%
0	0.1%	16.6%
0	0.1%	18.3%
0	0.1%	20.1%
0	0.1%	22%
0	0.1%	24%
0	0.1%	26.2%
0	0.1%	28.5%
0	0.1%	30.9%
0.001	0.1%	33.4%
0.001	0.1%	36%
0.001	0.1%	38.7%
0.001	0.1%	41.5%
0.001	0.1%	44.3%
0.001	0.1%	47.1%
0.001	0.1%	50%
0.001	0.1%	52.9%
0.001	0.1%	55.8%
0.001	0.1%	58.6%
0.002	0.1%	61.4%
0.002	0.1%	64%
0.002	0.1%	66.7%
0.002	0.1%	69.2%
0.003	0.1%	71.6%
0.003	0.1%	73.9%
0.003	0.1%	76%
0.004	0.1%	78.1%
0.004	0.1%	80%
0.004	0.1%	81.8%
0.005	0.1%	83.4%
0.006	0.1%	85%
0.006	0.1%	86.4%
0.007	0.1%	87.7%
0.008	0.1%	88.9%
0.009	0.1%	90%
0.01	0.1%	91%
0.011	0.1%	91.9%
0.013	0.1%	92.7%
0.014	0.1%	93.5%
0.016	0.1%	94.2%
0.018	0.1%	94.8%
0.02	0.1%	95.3%
0.022	0.1%	95.8%
0.025	0.1%	96.3%
0.028	0.1%	96.7%
0.032	0.1%	97%
0.036	0.1%	97.4%
0.04	0.1%	97.6%
0.045	0.1%	97.9%
0.05	0.1%	98.1%
0.056	0.1%	98.3%
0.063	0.1%	98.5%
0.071	0.1%	98.7%
0.079	0.1%	98.9%
0.089	0.1%	99%
0.1	0.1%	99.1%

Worked examples, recomputed

Every number the mechanisms quote for this relationship is recomputed from it, so none of them disagrees with it.

10⁻³ incidents, 1% false positives → precision ≈ 9%

Where it appears

Each mechanism states it in its own names. The general symbol each name stands for is shown with it.

Observability & Operations
What is the smallest description of a running system from which we can find a fault, fast enough to matter, at a cost we can still pay while the system is failing?
$Precision = 0.001/ (0.001 + 0.01) \approx 9%$
Read it in place →