Handle late data for each consumer
A late record is one whose event time is below the current watermark. Because the watermark is a guess made at a chosen percentile, lateness is not an anomaly: it is the residual you budgeted for. If you chose p99.9, you expected 0.1% of records to be late. There are exactly three things to do with one: drop it, divert it, or re-fire the window. Which does this consumer need?
You will end with for each consumer, what happens to a late record, how long window state is kept, and the accumulation mode its sink needs.
Decide
Does a missing late record change this consumer's answer in a way that matters?
A live operations view may tolerate it; a billing total may not.
Can you keep each window's state until the watermark passes its end plus the allowed lateness?
Why
Re-firing needs the window's state. With 1-minute windows and 10 minutes of allowed lateness, each key holds 1 + 10/1 = 11 windows at once; 20 minutes holds 21. For short windows, doubling the allowed lateness roughly doubles the memory.
Is the consumer's sink keyed, so a new result can overwrite the old one?
Why
The accumulation mode is a statement about the algebra of the sink. Getting it wrong gives double counting that stays invisible until someone compares totals with the source.
Where you can land
Drop, and count it
Use it when a late record does not change what this consumer decides.
What it costs: completeness, by exactly the late share. A silent drop turns a correctness failure into a quiet shift in the numbers; expose a count of late records dropped, and have someone watch it.
Divert to a side output
Use it when late records matter but window state cannot be kept long enough.
What it costs: a place for the late records and a person who looks at them, the same obligation as a dead-letter queue.
Re-fire with totals (accumulating)
Use it when the sink overwrites by key.
Each new firing emits the window's new total: 3,100, then 3,260.
What it costs: window state for the whole allowed lateness, and a keyed, mutable sink.
Re-fire with deltas (discarding)
Use it when the sink adds what it receives.
Each new firing emits only the change since the last one: 3,100, then +160.
What it costs: window state for the whole allowed lateness, and a sink that received every earlier firing exactly once. Emit totals into a sink that adds them and you get 3,100 + 3,260 = 6,360, nearly double.
Whichever you land on
Do not take exactly-once for completeness
Exactly-once processing is a statement about duplicates, not coverage. A pipeline can be exactly-once and still produce wrong totals because it dropped late data.
Label early results as early
If the window also fires before the watermark, the consumer must be told the number is provisional. A count that rises and then falls is not a pipeline bug; it is a consumer who was never told.