Varint and zigzag: where the bet wins and loses

Measure your machineNeeds Python

Problem

A varint stores an integer in 7-bit groups, one byte each, with a flag bit saying whether another byte follows: small numbers take one byte, a 64-bit number up to ten. It is a bet that values are small. Negative numbers break the bet: in two's complement −1 is all ones, the largest unsigned value, and costs ten bytes. Zigzag encoding maps signed to unsigned by interleaving (0, −1, 1, −2, 2 …) so that small magnitudes of either sign stay small.

Measure bytes per value for varint on four kinds of data, and see why zigzag is needed.

The question. How many bytes per value does varint take on each kind of data, and what does zigzag change?

Predict

For random 64-bit values, how many bytes per value does varint take?

Fewer than 8About 8, like fixed widthMore than 8

Run it

1Install uv once. It fetches Python and the lab's packages for you.
Terminal
```
curl -LsSf https://astral.sh/uv/install.sh | sh
```

2Get the script. It is the whole lab: read it, then download it.

varint-and-zigzag-where-the-bet.py39 linesDownload

View the code

varint-and-zigzag-where-the-bet.py

# /// script
# requires-python = ">=3.10"
# ///
# Copyright (c) 2026 Aayush Sharma · SPDX-License-Identifier: MIT
# https://thedataengineering.com/lab-code/LICENSE.txt
"""Varint and zigzag: where the bet pays and where it loses.

Bytes per value for varint on four kinds of data, with and without zigzag
for signed values, against a fixed 8-byte int64.
"""
import argparse, json, random

p = argparse.ArgumentParser(description=__doc__.splitlines()[0])
p.add_argument("--values", type=int, default=100_000, help="Values per set.")
p.add_argument("--seed", type=int, default=0, help="Random seed.")
p.add_argument("--json", help="Also write the results to this JSON file.")
args = p.parse_args()
random.seed(args.seed)
def varint_len(u):
    n = 1
    while u >= 0x80: u >>= 7; n += 1
    return n
zigzag = lambda n: ((n << 1) ^ (n >> 63)) & ((1 << 64) - 1)
twos = lambda n: n & ((1 << 64) - 1)
N = args.values
sets = {"small counts 0..100": [random.randrange(100) for _ in range(N)],
        "ids up to 10^9": [random.randrange(10**9) for _ in range(N)],
        "random 64-bit": [random.getrandbits(64) for _ in range(N)],
        "small signed -50..50": [random.randint(-50, 50) for _ in range(N)]}
rows = []
for name, v in sets.items():
    plain = sum(varint_len(twos(x)) for x in v) / N; zz = sum(varint_len(zigzag(x)) for x in v) / N
    rows.append({"data": name, "varint_bytes": plain, "zigzag_bytes": zz, "fixed_bytes": 8})
    print(f"{name:24s} varint {plain:5.2f} B   zigzag+varint {zz:5.2f} B   (fixed int64 = 8 B)")

if args.json:
    with open(args.json, "w") as f:
        json.dump({"tables": {"sets": rows}}, f)

3Run it in the folder you saved it to, with the defaults first. Then change a knob from the table below and run it again.
Terminal
```
uv run varint-and-zigzag-where-the-bet.py
uv run varint-and-zigzag-where-the-bet.py --help     # every knob
```

Knobs
Knob · default	What it changes
--values100,000	Values per set.
--seed0	Random seed.

Our run

Bytes per value

Bytes per value
Data	varint	zigzag + varint	fixed int64
small counts 0..100	1	1.36	8
ids up to 10^9	4.73	4.86	8
random 64-bit	9.49	9.5	8
small signed -50..50	5.46	1	8

Reference runApple M2 · 8 cores · 8 GB

Output · uv run varint-and-zigzag-where-the-bet.py

small counts 0..100      varint  1.00 B   zigzag+varint  1.36 B   (fixed int64 = 8 B)
ids up to 10^9           varint  4.73 B   zigzag+varint  4.86 B   (fixed int64 = 8 B)
random 64-bit            varint  9.49 B   zigzag+varint  9.50 B   (fixed int64 = 8 B)
small signed -50..50     varint  5.46 B   zigzag+varint  1.00 B   (fixed int64 = 8 B)

macOS 26.6.2 · Python 3.13.3 · run 2026-10-20

What we found

Every expectation held: small counts took 1 byte, IDs below 10⁹ 4.73 bytes, random 64-bit values 9.49 bytes (more than a fixed 8), and small signed numbers 5.46 bytes without zigzag against 1 with it. Zigzag is not free on unsigned data: small counts grew to 1.36 bytes, because zigzag doubles every positive value and pushes 64 to 100 into a second byte.

Read it

Varint is a bet on the distribution, and it pays only when the values are small. On uniformly random 64-bit values it loses to a fixed width; on signed data it needs zigzag. Formats that use it (Protocol Buffers, Avro, Thrift) let the schema choose: signed types that zigzag, unsigned types that do not, and fixed types for hashes and random IDs.