Model
The misses own the mean
A cache does not make things fast, it makes them sometimes fast, and the average is set by the failures.
- Hit rate
- fraction of requests served by the fast tier
- Tier latencies
- latency of the fast and the slow path
How it behaves
One input on the horizontal axis, the others held fixed.
mean latency (ms)share of the mean from misses (%)
Data table
| Hit rate h | series | Mean latency (ms) · misses' share (%) |
|---|---|---|
| 0.5 | mean latency (ms) | 25.1 |
| 0.506 | mean latency (ms) | 24.7 |
| 0.512 | mean latency (ms) | 24.4 |
| 0.519 | mean latency (ms) | 24.1 |
| 0.525 | mean latency (ms) | 23.8 |
| 0.531 | mean latency (ms) | 23.5 |
| 0.537 | mean latency (ms) | 23.2 |
| 0.544 | mean latency (ms) | 22.9 |
| 0.55 | mean latency (ms) | 22.6 |
| 0.556 | mean latency (ms) | 22.2 |
| 0.562 | mean latency (ms) | 21.9 |
| 0.569 | mean latency (ms) | 21.6 |
| 0.575 | mean latency (ms) | 21.3 |
| 0.581 | mean latency (ms) | 21 |
| 0.587 | mean latency (ms) | 20.7 |
| 0.594 | mean latency (ms) | 20.4 |
| 0.6 | mean latency (ms) | 20.1 |
| 0.606 | mean latency (ms) | 19.8 |
| 0.612 | mean latency (ms) | 19.4 |
| 0.619 | mean latency (ms) | 19.1 |
| 0.625 | mean latency (ms) | 18.8 |
| 0.631 | mean latency (ms) | 18.5 |
| 0.637 | mean latency (ms) | 18.2 |
| 0.643 | mean latency (ms) | 17.9 |
| 0.65 | mean latency (ms) | 17.6 |
| 0.656 | mean latency (ms) | 17.3 |
| 0.662 | mean latency (ms) | 17 |
| 0.668 | mean latency (ms) | 16.6 |
| 0.675 | mean latency (ms) | 16.3 |
| 0.681 | mean latency (ms) | 16 |
| 0.687 | mean latency (ms) | 15.7 |
| 0.693 | mean latency (ms) | 15.4 |
| 0.7 | mean latency (ms) | 15.1 |
| 0.706 | mean latency (ms) | 14.8 |
| 0.712 | mean latency (ms) | 14.5 |
| 0.718 | mean latency (ms) | 14.2 |
| 0.725 | mean latency (ms) | 13.8 |
| 0.731 | mean latency (ms) | 13.5 |
| 0.737 | mean latency (ms) | 13.2 |
| 0.743 | mean latency (ms) | 12.9 |
| 0.75 | mean latency (ms) | 12.6 |
| 0.756 | mean latency (ms) | 12.3 |
| 0.762 | mean latency (ms) | 12 |
| 0.768 | mean latency (ms) | 11.7 |
| 0.774 | mean latency (ms) | 11.4 |
| 0.781 | mean latency (ms) | 11 |
| 0.787 | mean latency (ms) | 10.7 |
| 0.793 | mean latency (ms) | 10.4 |
| 0.799 | mean latency (ms) | 10.1 |
| 0.806 | mean latency (ms) | 9.8 |
| 0.812 | mean latency (ms) | 9.49 |
| 0.818 | mean latency (ms) | 9.18 |
| 0.824 | mean latency (ms) | 8.86 |
| 0.831 | mean latency (ms) | 8.55 |
| 0.837 | mean latency (ms) | 8.24 |
| 0.843 | mean latency (ms) | 7.93 |
| 0.849 | mean latency (ms) | 7.62 |
| 0.856 | mean latency (ms) | 7.31 |
| 0.862 | mean latency (ms) | 7 |
| 0.868 | mean latency (ms) | 6.69 |
| 0.874 | mean latency (ms) | 6.37 |
| 0.88 | mean latency (ms) | 6.06 |
| 0.887 | mean latency (ms) | 5.75 |
| 0.893 | mean latency (ms) | 5.44 |
| 0.899 | mean latency (ms) | 5.13 |
| 0.905 | mean latency (ms) | 4.82 |
| 0.912 | mean latency (ms) | 4.51 |
| 0.918 | mean latency (ms) | 4.2 |
| 0.924 | mean latency (ms) | 3.88 |
| 0.93 | mean latency (ms) | 3.57 |
| 0.937 | mean latency (ms) | 3.26 |
| 0.943 | mean latency (ms) | 2.95 |
| 0.949 | mean latency (ms) | 2.64 |
| 0.955 | mean latency (ms) | 2.33 |
| 0.962 | mean latency (ms) | 2.02 |
| 0.968 | mean latency (ms) | 1.71 |
| 0.974 | mean latency (ms) | 1.39 |
| 0.98 | mean latency (ms) | 1.08 |
| 0.987 | mean latency (ms) | 0.772 |
| 0.993 | mean latency (ms) | 0.461 |
| 0.999 | mean latency (ms) | 0.15 |
| 0.5 | share of the mean from misses (%) | 99.8 |
| 0.506 | share of the mean from misses (%) | 99.8 |
| 0.512 | share of the mean from misses (%) | 99.8 |
| 0.519 | share of the mean from misses (%) | 99.8 |
| 0.525 | share of the mean from misses (%) | 99.8 |
| 0.531 | share of the mean from misses (%) | 99.8 |
| 0.537 | share of the mean from misses (%) | 99.8 |
| 0.544 | share of the mean from misses (%) | 99.8 |
| 0.55 | share of the mean from misses (%) | 99.8 |
| 0.556 | share of the mean from misses (%) | 99.8 |
| 0.562 | share of the mean from misses (%) | 99.7 |
| 0.569 | share of the mean from misses (%) | 99.7 |
| 0.575 | share of the mean from misses (%) | 99.7 |
| 0.581 | share of the mean from misses (%) | 99.7 |
| 0.587 | share of the mean from misses (%) | 99.7 |
| 0.594 | share of the mean from misses (%) | 99.7 |
| 0.6 | share of the mean from misses (%) | 99.7 |
| 0.606 | share of the mean from misses (%) | 99.7 |
| 0.612 | share of the mean from misses (%) | 99.7 |
| 0.619 | share of the mean from misses (%) | 99.7 |
| 0.625 | share of the mean from misses (%) | 99.7 |
| 0.631 | share of the mean from misses (%) | 99.7 |
| 0.637 | share of the mean from misses (%) | 99.6 |
| 0.643 | share of the mean from misses (%) | 99.6 |
| 0.65 | share of the mean from misses (%) | 99.6 |
| 0.656 | share of the mean from misses (%) | 99.6 |
| 0.662 | share of the mean from misses (%) | 99.6 |
| 0.668 | share of the mean from misses (%) | 99.6 |
| 0.675 | share of the mean from misses (%) | 99.6 |
| 0.681 | share of the mean from misses (%) | 99.6 |
| 0.687 | share of the mean from misses (%) | 99.6 |
| 0.693 | share of the mean from misses (%) | 99.5 |
| 0.7 | share of the mean from misses (%) | 99.5 |
| 0.706 | share of the mean from misses (%) | 99.5 |
| 0.712 | share of the mean from misses (%) | 99.5 |
| 0.718 | share of the mean from misses (%) | 99.5 |
| 0.725 | share of the mean from misses (%) | 99.5 |
| 0.731 | share of the mean from misses (%) | 99.5 |
| 0.737 | share of the mean from misses (%) | 99.4 |
| 0.743 | share of the mean from misses (%) | 99.4 |
| 0.75 | share of the mean from misses (%) | 99.4 |
| 0.756 | share of the mean from misses (%) | 99.4 |
| 0.762 | share of the mean from misses (%) | 99.4 |
| 0.768 | share of the mean from misses (%) | 99.3 |
| 0.774 | share of the mean from misses (%) | 99.3 |
| 0.781 | share of the mean from misses (%) | 99.3 |
| 0.787 | share of the mean from misses (%) | 99.3 |
| 0.793 | share of the mean from misses (%) | 99.2 |
| 0.799 | share of the mean from misses (%) | 99.2 |
| 0.806 | share of the mean from misses (%) | 99.2 |
| 0.812 | share of the mean from misses (%) | 99.1 |
| 0.818 | share of the mean from misses (%) | 99.1 |
| 0.824 | share of the mean from misses (%) | 99.1 |
| 0.831 | share of the mean from misses (%) | 99 |
| 0.837 | share of the mean from misses (%) | 99 |
| 0.843 | share of the mean from misses (%) | 98.9 |
| 0.849 | share of the mean from misses (%) | 98.9 |
| 0.856 | share of the mean from misses (%) | 98.8 |
| 0.862 | share of the mean from misses (%) | 98.8 |
| 0.868 | share of the mean from misses (%) | 98.7 |
| 0.874 | share of the mean from misses (%) | 98.6 |
| 0.88 | share of the mean from misses (%) | 98.5 |
| 0.887 | share of the mean from misses (%) | 98.5 |
| 0.893 | share of the mean from misses (%) | 98.4 |
| 0.899 | share of the mean from misses (%) | 98.2 |
| 0.905 | share of the mean from misses (%) | 98.1 |
| 0.912 | share of the mean from misses (%) | 98 |
| 0.918 | share of the mean from misses (%) | 97.8 |
| 0.924 | share of the mean from misses (%) | 97.6 |
| 0.93 | share of the mean from misses (%) | 97.4 |
| 0.937 | share of the mean from misses (%) | 97.1 |
| 0.943 | share of the mean from misses (%) | 96.8 |
| 0.949 | share of the mean from misses (%) | 96.4 |
| 0.955 | share of the mean from misses (%) | 95.9 |
| 0.962 | share of the mean from misses (%) | 95.2 |
| 0.968 | share of the mean from misses (%) | 94.3 |
| 0.974 | share of the mean from misses (%) | 93 |
| 0.98 | share of the mean from misses (%) | 91 |
| 0.987 | share of the mean from misses (%) | 87.2 |
| 0.993 | share of the mean from misses (%) | 78.5 |
| 0.999 | share of the mean from misses (%) | 33.4 |
Worked examples, recomputed
Every number the mechanisms quote for this relationship is recomputed from it, so none of them disagrees with it.
- 40% in front of a 50 ms store → 19.96 ms saved
- 95% in front of a 0.2 ms probe → 0.19 ms saved
- pooling with a slow origin gains ≈ 7.2 ms
- remote round trips cost ≈ 0.43 ms
- with a 2 ms origin the gain is ≈ 0.22 ms
- 99% cost 1, 1% cost 500 → mean 5.99, the tail is 83% of it
- quarantining the 1% of ₹1,950 orders hides 4.2% of revenue (true average ₹465)
- 990 at 5 ms, 10 at 700 ms → mean 11.95 ms
Where it appears
Each mechanism states it in its own names. The general symbol each name stands for is shown with it.
- Caching
What happens when we deliberately keep a second copy of data that no protocol forces to be correct?