Skip to content
The Data EngineeringThe Data EngineeringBeta
About
Main content

Throughput at an SLO, and goodput

Why is capacity the rate at which the SLO still holds, and why count what clients received?

Throughput at a fixed latency SLO

A "peak " number is a measurement taken past the , where has already diverged. (The knee is the beyond which latency rises sharply; Load generation showed how to find it.) So it is a capacity claim about a regime you would never run in. Comparing two systems on it compares their behaviour in a regime where neither is deployed. This is the problem with the vendor slide "180,000 holds/s".

The defensible quantity is the maximum arrival rate at which the latency is still met (an SLO is a service level objective, for example " below 50 ms"). This is throughput under a constraint. It is one point on the frontier you found in the sweep, not an independent number. Stated this way, the comparison is honest: two systems are compared at equal quality of service, not at equal misery.

Capacity at the SLO: Engine A about 80,000/s, Tessera about 120,000/s. A "peak" figure would instead report whatever each system pushes through while already failing its SLO, which says nothing about on-sale day. (These numbers are illustrations.)

This also changes what a result is. A result is a surface over (arrival rate, latency , cost, error rate). A single number is one point on that surface, chosen by whoever reports it. This is the principle "whoever chooses the conditions chooses the result; disclosure is the only audit surface". The choice of which point to report is where the persuasion happens, and "peak throughput" is the most flattering point available.

Goodput vs. throughput

Throughput counts work done. Goodput counts work that was useful to someone. They diverge exactly under stress, so throughput is most misleading when you most need it.

Sources of divergence:

  • Retries that reprocess the same logical request.
  • Speculative and hedged execution that does deliberate duplicate work.
  • Responses returned after the client has already given up.
  • Results thrown away by a failed transaction.
  • Recomputation after a lost task.

In a (a failure mode where clients retry failed requests, which adds load, which causes more failures), throughput can be at its maximum while is close to zero. The system is perfectly busy doing nothing useful, and every dashboard reads healthy.

The measurement rule: count completions at the client, against the original logical request, not at the server against units of work. A server counter cannot tell a first attempt from a fourth. In cost terms, the ratio goodput/throughput is the fraction of your spend that bought something. So it is, unusually, one metric that is both an operational number and a financial number.

This is the principle "the unit of measurement must match the unit of value". Fans want seats, not server completions.