Idle: the most misunderstood cost

What sets the idle you must pay for, and what does serverless really sell?

Idle capacity is the cost most often treated as pure waste. It is not a defect in the model. Occupancy is the product, so idle is what you bought. Decompose it, because each cause has a different fix, and only some are fixable.

The floor, set by queueing. A latency-sensitive system must keep utilization well below 1 (recall R = S ÷ (1 − ρ)). A target of ρ ≈ 0.6–0.7 means 30–40% idle is mandatory. Buying it back destroys the SLO.
Burstiness. If you provision for the peak and the peak is 3× the mean, you multiply again.
Granularity. You need 1.1 nodes and you can only buy whole nodes, so you buy 2.

The combined multiplier is roughly (peak ÷ mean) ÷ ρ_target.

RelationshipWhat variance costs in money →

(peak \div mean) \div ρ_{target}

$ρ_{target}$: kept well below 1 for latency (0.6–0.7)

ρ target 0.65ρ target 0.9 (batch)

Data table

Provisioned ÷ mean demand, by burstiness
Peak ÷ mean	series	Provisioned ÷ mean demand
1	ρ target 0.65	1.54×
1.11	ρ target 0.65	1.71×
1.23	ρ target 0.65	1.88×
1.34	ρ target 0.65	2.06×
1.45	ρ target 0.65	2.23×
1.56	ρ target 0.65	2.4×
1.68	ρ target 0.65	2.58×
1.79	ρ target 0.65	2.75×
1.9	ρ target 0.65	2.92×
2.01	ρ target 0.65	3.1×
2.13	ρ target 0.65	3.27×
2.24	ρ target 0.65	3.44×
2.35	ρ target 0.65	3.62×
2.46	ρ target 0.65	3.79×
2.58	ρ target 0.65	3.96×
2.69	ρ target 0.65	4.13×
2.8	ρ target 0.65	4.31×
2.91	ρ target 0.65	4.48×
3.02	ρ target 0.65	4.65×
3.14	ρ target 0.65	4.83×
3.25	ρ target 0.65	5×
3.36	ρ target 0.65	5.17×
3.48	ρ target 0.65	5.35×
3.59	ρ target 0.65	5.52×
3.7	ρ target 0.65	5.69×
3.81	ρ target 0.65	5.87×
3.92	ρ target 0.65	6.04×
4.04	ρ target 0.65	6.21×
4.15	ρ target 0.65	6.38×
4.26	ρ target 0.65	6.56×
4.38	ρ target 0.65	6.73×
4.49	ρ target 0.65	6.9×
4.6	ρ target 0.65	7.08×
4.71	ρ target 0.65	7.25×
4.83	ρ target 0.65	7.42×
4.94	ρ target 0.65	7.6×
5.05	ρ target 0.65	7.77×
5.16	ρ target 0.65	7.94×
5.28	ρ target 0.65	8.12×
5.39	ρ target 0.65	8.29×
5.5	ρ target 0.65	8.46×
5.61	ρ target 0.65	8.63×
5.72	ρ target 0.65	8.81×
5.84	ρ target 0.65	8.98×
5.95	ρ target 0.65	9.15×
6.06	ρ target 0.65	9.33×
6.17	ρ target 0.65	9.5×
6.29	ρ target 0.65	9.67×
6.4	ρ target 0.65	9.85×
6.51	ρ target 0.65	10×
6.63	ρ target 0.65	10.2×
6.74	ρ target 0.65	10.4×
6.85	ρ target 0.65	10.5×
6.96	ρ target 0.65	10.7×
7.08	ρ target 0.65	10.9×
7.19	ρ target 0.65	11.1×
7.3	ρ target 0.65	11.2×
7.41	ρ target 0.65	11.4×
7.53	ρ target 0.65	11.6×
7.64	ρ target 0.65	11.8×
7.75	ρ target 0.65	11.9×
7.86	ρ target 0.65	12.1×
7.97	ρ target 0.65	12.3×
8.09	ρ target 0.65	12.4×
8.2	ρ target 0.65	12.6×
8.31	ρ target 0.65	12.8×
8.43	ρ target 0.65	13×
8.54	ρ target 0.65	13.1×
8.65	ρ target 0.65	13.3×
8.76	ρ target 0.65	13.5×
8.88	ρ target 0.65	13.7×
8.99	ρ target 0.65	13.8×
9.1	ρ target 0.65	14×
9.21	ρ target 0.65	14.2×
9.32	ρ target 0.65	14.3×
9.44	ρ target 0.65	14.5×
9.55	ρ target 0.65	14.7×
9.66	ρ target 0.65	14.9×
9.78	ρ target 0.65	15×
9.89	ρ target 0.65	15.2×
10	ρ target 0.65	15.4×
1	ρ target 0.9 (batch)	1.11×
1.11	ρ target 0.9 (batch)	1.24×
1.23	ρ target 0.9 (batch)	1.36×
1.34	ρ target 0.9 (batch)	1.49×
1.45	ρ target 0.9 (batch)	1.61×
1.56	ρ target 0.9 (batch)	1.74×
1.68	ρ target 0.9 (batch)	1.86×
1.79	ρ target 0.9 (batch)	1.99×
1.9	ρ target 0.9 (batch)	2.11×
2.01	ρ target 0.9 (batch)	2.24×
2.13	ρ target 0.9 (batch)	2.36×
2.24	ρ target 0.9 (batch)	2.49×
2.35	ρ target 0.9 (batch)	2.61×
2.46	ρ target 0.9 (batch)	2.74×
2.58	ρ target 0.9 (batch)	2.86×
2.69	ρ target 0.9 (batch)	2.99×
2.8	ρ target 0.9 (batch)	3.11×
2.91	ρ target 0.9 (batch)	3.24×
3.02	ρ target 0.9 (batch)	3.36×
3.14	ρ target 0.9 (batch)	3.49×
3.25	ρ target 0.9 (batch)	3.61×
3.36	ρ target 0.9 (batch)	3.74×
3.48	ρ target 0.9 (batch)	3.86×
3.59	ρ target 0.9 (batch)	3.99×
3.7	ρ target 0.9 (batch)	4.11×
3.81	ρ target 0.9 (batch)	4.24×
3.92	ρ target 0.9 (batch)	4.36×
4.04	ρ target 0.9 (batch)	4.49×
4.15	ρ target 0.9 (batch)	4.61×
4.26	ρ target 0.9 (batch)	4.74×
4.38	ρ target 0.9 (batch)	4.86×
4.49	ρ target 0.9 (batch)	4.99×
4.6	ρ target 0.9 (batch)	5.11×
4.71	ρ target 0.9 (batch)	5.24×
4.83	ρ target 0.9 (batch)	5.36×
4.94	ρ target 0.9 (batch)	5.49×
5.05	ρ target 0.9 (batch)	5.61×
5.16	ρ target 0.9 (batch)	5.74×
5.28	ρ target 0.9 (batch)	5.86×
5.39	ρ target 0.9 (batch)	5.99×
5.5	ρ target 0.9 (batch)	6.11×
5.61	ρ target 0.9 (batch)	6.24×
5.72	ρ target 0.9 (batch)	6.36×
5.84	ρ target 0.9 (batch)	6.49×
5.95	ρ target 0.9 (batch)	6.61×
6.06	ρ target 0.9 (batch)	6.74×
6.17	ρ target 0.9 (batch)	6.86×
6.29	ρ target 0.9 (batch)	6.99×
6.4	ρ target 0.9 (batch)	7.11×
6.51	ρ target 0.9 (batch)	7.24×
6.63	ρ target 0.9 (batch)	7.36×
6.74	ρ target 0.9 (batch)	7.49×
6.85	ρ target 0.9 (batch)	7.61×
6.96	ρ target 0.9 (batch)	7.74×
7.08	ρ target 0.9 (batch)	7.86×
7.19	ρ target 0.9 (batch)	7.99×
7.3	ρ target 0.9 (batch)	8.11×
7.41	ρ target 0.9 (batch)	8.24×
7.53	ρ target 0.9 (batch)	8.36×
7.64	ρ target 0.9 (batch)	8.49×
7.75	ρ target 0.9 (batch)	8.61×
7.86	ρ target 0.9 (batch)	8.74×
7.97	ρ target 0.9 (batch)	8.86×
8.09	ρ target 0.9 (batch)	8.99×
8.2	ρ target 0.9 (batch)	9.11×
8.31	ρ target 0.9 (batch)	9.24×
8.43	ρ target 0.9 (batch)	9.36×
8.54	ρ target 0.9 (batch)	9.49×
8.65	ρ target 0.9 (batch)	9.61×
8.76	ρ target 0.9 (batch)	9.74×
8.88	ρ target 0.9 (batch)	9.86×
8.99	ρ target 0.9 (batch)	9.99×
9.1	ρ target 0.9 (batch)	10.1×
9.21	ρ target 0.9 (batch)	10.2×
9.32	ρ target 0.9 (batch)	10.4×
9.44	ρ target 0.9 (batch)	10.5×
9.55	ρ target 0.9 (batch)	10.6×
9.66	ρ target 0.9 (batch)	10.7×
9.78	ρ target 0.9 (batch)	10.9×
9.89	ρ target 0.9 (batch)	11×
10	ρ target 0.9 (batch)	11.1×

Principle: Variance costs more than volume →

What serverless really sells

This explains why serverless can make sense even when its unit prices are several times higher. The product is not compute. It is statistical multiplexing of your duty cycle: the vendor combines many customers so that their peaks do not coincide, and keeps the difference between your multiplier and theirs.

The hidden assumption is that tenants are independent, which nobody can verify and which is usually false at the edges. Everyone's midnight cron job fires at midnight, and that is exactly when capacity problems cluster.