Time & Temporal Semantics
When am I allowed to act on data that may still be incomplete, and what do I owe the reader if I was wrong?
Who this mechanism is for: a developer who writes code, builds data pipelines or dashboards, and knows basic SQL, but has never studied stream processing, watermarks or temporal databases. You do not need any background in distributed systems. Every technical term is explained the first time it appears, and again in the glossary at the end. Small code snippets are in Python, SQL and shell.
The story we will follow
Kavya is a data engineer at Kitebox Deliveries, a company that delivers parcels in several cities. Every courier carries a phone app. The app sends small messages called events: "parcel picked up", "parcel scanned at depot", "parcel delivered". Kitebox Deliveries stores these events and computes numbers such as "deliveries per depot per hour". Her manager, Rohan, reads these numbers every morning, and the finance team uses them to pay couriers.
Over one busy quarter Kavya will meet six problems, one at the start of each article below. Each problem looks like a bug in her code. None of them is. Each one comes from a fact about time that you cannot engineer away. At the end of each article, we go back and solve her problem with the mechanisms of that part. The last part (What time forbids) puts all the limits in one list.
This is the main idea of this mechanism:
How this mechanism is organized
- 1
Three clocks
Which "time" is on a record, who wrote it, and how far can we trust it?
CaseSame data, same code, a different count
- 2
Watermarks
How can a pipeline declare "everything before time T has arrived" when it cannot know?
CaseA window that never closes
- 3
Late data and triggers
What happens to records that arrive after the declaration, and when should results be emitted?
Case3,100 sent to finance, 3,260 the next morning
- 4
Partitioning and corrections
Which clock should decide where data is stored, and how do we publish corrections?
CaseNine days of folders for one day's answer
- 5
Bitemporal data and joins
How do we store the fact that our knowledge of the past changes?
CaseWhat did we believe on 10 March?
- 6
Calendars, clocks and expiry
Why do time zones, wall clocks and TTLs break systems?
CaseParcels delivered twice on the last Sunday of October
- 7
What time forbids
Which things can never be had together, and what does each way out cost?
CaseOne number: fast, complete and unchanging?
After that: a summary, hands-on labs, interview questions and a glossary.
What do I do, and what does each option cost?
Can I see it myself?
Measure your skew distribution and price δ
A million heavy-tailed arrival skews: p50 to p99.99, and for each used as the watermark delay, the share of late records and the extra wait.
MeasurePython with NumPyWatermark with an idle partition
Three partitions, one goes quiet for 10 minutes and then sends its backlog: window closing delay and late events with and without an idleness timeout.
SimulatePythonRetained-window memory
Tumbling windows of size W kept until the watermark passes their end plus allowed lateness L: windows held per key against 1 + L/W.
SimulatePythonTrigger mode versus sink
A window fires twice (3,100, then 3,260) in accumulating or discarding mode, into an overwriting sink or an additive counter.
SimulatePythonBitemporal table in SQLite or Postgres
A rate table with valid time and transaction time in SQLite: a correction, two as-of questions, and the same history with one updated_at column.
MeasurePythonEvent time versus availability time
Features with available_time = valid_time + 9 h: how many events get a different value when joined on availability, and what that does to a model.
MeasurePython with pandas, scikit-learnLeakage inflates the score (naive join versus as-of join)
20,000 deliveries with a drifting courier speed feature: a naive join on the latest feature value against an as-of join, scored offline and on a later period.
MeasurePython with pandas, scikit-learnThe daylight-saving key bug
02:30 on 26 October 2025 happens twice in Berlin: two instants, one local text. Then month arithmetic in two orders.
MeasurePythonReplay determinism test
A transformation that reads current_timestamp, run twice and "tomorrow"; then with a run_date parameter. Output hashes compared.
MeasurePython
Can I reason about it under questioning?
Interview questions
Short questions on time in data systems, such as event and processing time, watermarks, late data and triggers, corrections, bitemporal data, time zones and replay, each with an answer you could give aloud and a link to the full explanation.
Scenarios
Longer problems on time in data pipelines, of the kind asked in senior interviews: each one estimated with numbers, diagnosed, and answered with its trade-offs.
The tensions this mechanism takes a position on. Each recurs across mechanisms.
Space vs. time
For example: compression, indexing, materialization, caching.
Mutability vs. immutability
In-place update vs. append-only + GC; buys reproducibility and cheap concurrency, pays in space and janitorial work.
Early binding vs. late binding
When structure is committed; determines where the failure surfaces, at write time or read time.
Exactness vs. approximation
For example: sketches, sampling, ANN.
Fail-fast vs. fail-degraded
Reject or continue when something is wrong.
Ideas this mechanism meets that keep coming back, mechanism after mechanism.
Costs are conserved
Costs are conserved, never removed.
Variance costs more than volume
Variance costs more than volume.
The algebra decides
What can be skipped, pushed down, parallelized, merged or precomputed is determined by the algebra of the function, not the layout of the data.
Immutable storage grows indirection
An immutable substrate grows an indirection layer plus a collector.
Absence is not evidence
Absence of an event is not absence of a failure.