Gap filling, and what time series forbids

Why is every gap fill an assumption, and what can time-series storage never give you?

Gap filling and interpolation

A gap in a series is ambiguous in a way that a missing row in a table is not. It can mean any of four things: the value did not change and nothing was emitted; the sensor was down; the network dropped the data; or the series did not exist yet. The stored representation is identical in all four cases, and the correct query behaviour differs in all four.

This is the family's genuinely informational constraint. The data does not record why it is absent, so any interpolation is an assumption injected by the reader.

Fill method	Assumption	Right for	Failure
Last value carried forward	the value is a step function	state (e.g. "courier is online")	wrong for events
Linear interpolation	change is smooth	smooth physical signals	invents data that was never measured
Zero fill	absence means zero	counts of events	turns an outage into a reported value of zero, so a dashboard says "traffic is fine" while the collector is down

Two things follow.

Interpolation is read-time semantics applied to write-time ambiguity, and it is unverifiable. The system cannot check whether the assumption holds. It can only apply it consistently. The only real fix is at the source: emit explicitly, and distinguish "no change" from "no data". That is a contract between producer and consumer, not a query feature. This is the reproducibility rule: where the truth is unrecoverable, substitute a declared convention and make it reproducible.

Alignment is the hidden second problem. Joining two series requires their timestamps to line up, and independently sampled series never do. So every cross-series operation implicitly resamples or does an as-of join (for each row on the left, take the latest row on the right at or before that time). This is a routine operation here, not an exotic one. It is the same primitive training sets need for point-in-time correctness, for the same reason: when two sources have independent clocks, every join between them is a temporal nearest-match, and the only question is which direction you are allowed to look.

Looking forward is interpolation. Looking backward is correct. Looking forward by accident is future leakage.

One line: a gap does not record why it is a gap, so every fill is an unverifiable assumption. The only real fix is a write-time contract.

What time series forbids

You cannot add an unbounded label without unbounded cost, because series overhead is per series and incompressible.
You cannot roll up an aggregate whose algebra does not compose.
You cannot recover resolution you discarded, or answer a question whose sketch you did not store.
You cannot randomly update a chained-delta chunk.
You cannot distinguish the four meanings of a gap from the data alone.

The escapes and their prices

Escape	Price
Specialized encoding gives about an order of magnitude	immutable chunks, and a ratio that depends on how regular your signal is
Precomputed rollups give four orders of magnitude on read	write amplification, and the decision of which aggregates to keep
Mergeable sketches give composability for quantiles and distinct counts	exactness, with no refine stage available
Resolution tiering gives flat storage growth	irreversibly, the future questions
Cardinality quotas give survival	query flexibility, enforced against the person who wanted it
Interpolation gives a usable result from an ambiguous record	an assumption nobody can verify

Case, resolvedConstructed case · Amara at Haatly

One new label, and the metrics store fell over

A user_id label made the metrics store run out of memory in twenty minutes, raw points cost about 1 TB a month though most repeated the last value, and hourly averages could not answer a p99 question a year later. Where it started: Time series →

The memory outage. The user_id label multiplied 60 series into 120 million (about 360 GB of per-series overhead at 3 KB each). Amara removes the label, adds a write-time quota (maximum 10,000 series per metric, reject labels with unbounded values), and moves per-user debugging to logs and traces, where rows add instead of multiply.

The 1 TB per month. She turns on delta-of-delta timestamps and XOR value compression. Courier GPS timestamps arrive every second with almost no jitter, so most timestamps cost 1 bit. The 1 TB becomes about 94 GB. She also sets retention tiers: raw for 7 days, one-minute for 30 days, one-hour for a year, one-day for five years.

The missing p99. The hourly average could never produce a p99, because percentiles do not compose. Going forward she stores a t-digest sketch per hour and per minute, plus min, max, sum and count, so quantiles roll up. The old quarter's p99 is gone for good (the decision was irreversible), and she writes that down as a lesson. She also asks the courier app to send an explicit "offline" event, so that a gap in GPS points can be told apart from a courier who did not move.