Find the backpressure culprit
When an operator is slower than the one before it, the engine slows the source, hop by hop, through bounded buffers. That keeps the job correct, and it makes every operator upstream of the bottleneck report backpressure. The red operators are the ones being held back, not the one doing the holding. Which operator is the real culprit?
You will end with the one operator that is slow by itself, why it is slow, and the fix that belongs to it.
Work in order and stop at the first hit.
Diagnose
Read the graph from the sink backwards
The culprit is the first operator, reading downstream, that is not itself backpressured: its output is not blocked by anything after it, so it is slow on its own. In source → parse → fraud → join → billing sink with the first four red, the culprit is the sink.
If the culprit is a sink, fix the system it writes to
The fix is at the database or service behind the sink, not in the red operators upstream. Adding parallelism to them changes nothing: the job's throughput is the minimum over all its operators.
If one instance of the culprit is hot and the rest are idle, look for one key
A single key's state is indivisible: all its records go to one instance. If one key carries 30% of the traffic, the job can never exceed about 3.3 times one instance's capacity, whether it has 8 instances or 500. Salting the key and combining partial results works only for associative and commutative aggregates (sum, count, max), and not for joins or anything that depends on order.
If the culprit is stateful, check its state backend
Write stalls in the state backend appear as backpressure in the topology. Compaction that has fallen behind under sustained writes competes with processing for the same disk and CPU.
Check whether checkpoints have slowed with the backlog
Aligned checkpoint barriers travel at the speed of the data, so under backpressure checkpoints slow exactly when the job is struggling, and the recovery point stops advancing. Unaligned checkpoints stay short, at the cost of saving in-flight data.
Check the headroom
A job running at 95% of capacity turns every burst into growing lag: response time grows as 1 ÷ (1 − ρ). If no single operator is broken, the job is simply too close to its capacity.
Check that the log can hold the lag
Backpressure is safe only because the durable log upstream absorbs it. Confirm that its retention covers the lag you now have, or the backlog becomes data loss.