Inference is a function of the sample

Measure your machineNeeds Python

Problem

A schema inferred from data is a function of the sample it was inferred from. If a field is an integer for the first 15,000 records and a string afterwards, a reader that samples the first 1,000 infers an integer and fails, or worse, coerces, when it meets the strings; a reader that samples more, or samples a shuffled file, infers a string. The same file gets different schemas in different jobs. A frozen schema, compared with each inference, turns those differences into alerts.

Generate a JSON-lines file of 20,000 records in which postal_code is an integer in the first 15,000 and a string such as "SW1A 1AA" afterwards. Infer the schema using a sample of the first 1,000 rows, then the first 18,000 rows, then all rows. Then shuffle the file and repeat. Add a frozen-schema drift check.

The question. Which type does each sample infer for postal_code, and does the drift check catch the difference?

Predict

Inferring from the first 1,000 rows in file order (string postal codes start at row 15,000), what type does postal_code get?

integerstring

Run it

1Install uv once. It fetches Python and the lab's packages for you.
Terminal
```
curl -LsSf https://astral.sh/uv/install.sh | sh
```

2Get the script. It is the whole lab: read it, then download it.

inference-is-a-function-of-the.py48 linesDownload

View the code

inference-is-a-function-of-the.py

# /// script
# requires-python = ">=3.10"
# ///
# Copyright (c) 2026 Aayush Sharma · SPDX-License-Identifier: MIT
# https://thedataengineering.com/lab-code/LICENSE.txt
"""Inference is a function of the sample.

A JSON-lines file of 20,000 records whose postal_code is an integer in the
first 15,000 and a string such as "SW1A 1AA" after that. Infers the schema
from the first 1,000 rows, the first 18,000 and all of them, in file order
and shuffled; then compares each inference with a frozen schema (a drift
check) and counts the alerts.
"""
import argparse, json, random

p = argparse.ArgumentParser(description=__doc__.splitlines()[0])
p.add_argument("--rows", type=int, default=20_000, help="Records.")
p.add_argument("--switch", type=int, default=15_000, help="Record from which postal_code is a string.")
p.add_argument("--seed", type=int, default=0, help="Random seed.")
p.add_argument("--json", help="Also write the results to this JSON file.")
args = p.parse_args()
random.seed(args.seed)

lines = [json.dumps({"id": i, "postal_code": random.randrange(10000, 99999) if i < args.switch else f"SW{random.randrange(1, 9)}A {random.randrange(1, 9)}AA",
                     "weight": round(random.uniform(0.1, 30), 2)}) for i in range(args.rows)]
TYPES = {int: "integer", float: "number", str: "string", bool: "boolean", type(None): "null"}
def infer(sample):
    out = {}
    for line in sample:
        for k, v in json.loads(line).items():
            t = TYPES[type(v)]; prev = out.get(k)
            out[k] = t if prev in (None, t) else ("number" if {prev, t} == {"integer", "number"} else "string")   # widen on conflict
    return out
FROZEN = {"id": "integer", "postal_code": "string", "weight": "number"}             # the contract, agreed and versioned
def drift(schema): return [f"{k}: {schema.get(k)} (expected {v})" for k, v in FROZEN.items() if schema.get(k) != v]
rows = []
shuffled = lines[:]; random.shuffle(shuffled)
for order, data in (("file order", lines), ("shuffled", shuffled)):
    for n in (1000, 18000, args.rows):
        sch = infer(data[:n]); alerts = drift(sch)
        rows.append({"order": order, "sample": n, "postal_code": sch["postal_code"], "weight": sch["weight"], "alerts": len(alerts), "alert": "; ".join(alerts) or "none"})
summary = {"distinct_inferences": len({(r["postal_code"], r["weight"]) for r in rows})}
for r in rows: print(f"{r['order']:<10} first {r['sample']:>6,} rows: postal_code {r['postal_code']:<8} weight {r['weight']:<7} drift alerts: {r['alert']}")

if args.json:
    with open(args.json, "w") as f:
        json.dump({"summary": summary, "tables": {"inferences": rows}}, f)

3Run it in the folder you saved it to, with the defaults first. Then change a knob from the table below and run it again.
Terminal
```
uv run inference-is-a-function-of-the.py
uv run inference-is-a-function-of-the.py --help     # every knob
```

Knobs
Knob · default	What it changes
--rows20,000	Records.
--switch15,000	Record from which postal_code is a string.
--seed0	Random seed.

Our run

Six inferences of one file

Six inferences of one file
Order	Sample	postal_code	Drift alert
file order	1K	integer	postal_code: integer (expected string)
file order	18K	string	none
file order	20K	string	none
shuffled	1K	string	none
shuffled	18K	string	none
shuffled	20K	string	none

Reference runApple M2 · 8 cores · 8 GB

Output · uv run inference-is-a-function-of-the.py

file order first  1,000 rows: postal_code integer  weight number  drift alerts: postal_code: integer (expected string)
file order first 18,000 rows: postal_code string   weight number  drift alerts: none
file order first 20,000 rows: postal_code string   weight number  drift alerts: none
shuffled   first  1,000 rows: postal_code string   weight number  drift alerts: none
shuffled   first 18,000 rows: postal_code string   weight number  drift alerts: none
shuffled   first 20,000 rows: postal_code string   weight number  drift alerts: none

macOS 26.6.2 · Python 3.13.3 · run 2026-10-20

What we found

In file order, the first 1,000 rows inferred postal_code as integer and the drift check raised an alert (postal_code: integer (expected string)); 18,000 and 20,000 rows inferred string. Shuffled, even 1,000 rows inferred string, because strings appeared in the sample. The file produced 2 distinct schemas depending on how it was read.

Read it

Inference answers "what does this sample look like", and the answer moves with the sample. Use it to propose a schema, then freeze it, and check every new batch against the frozen version, so that a type change is an alert at ingestion rather than a parse failure, or a silent coercion, three jobs downstream.