Skip to content
About
Main content

One old snapshot pins the version space

Run a real systemNeeds Docker, Python

Problem

Under an update leaves the old row version behind, and removes it once no can see it. A single snapshot opened long ago (a Repeatable Read transaction, a forgotten session, a long report) holds the horizon back for the whole database: every version created since cannot be removed, and tables grow with dead versions for as long as it stays open.

On PostgreSQL, create a table with 1 million rows and run random single-row updates from 8 clients for two minutes with autovacuum on. Record pg_total_relation_size and n_dead_tup. Repeat, but first open another session with BEGIN ISOLATION LEVEL REPEATABLE READ; SELECT count(*) ...; and leave it open. Then the old session, run VACUUM, and watch n_dead_tup drop.

The question. How much do the table and its dead tuples grow while one old snapshot is open, and what happens when it closes?


Predict

With one Repeatable Read snapshot left open during about a million updates, how large does the 142 MB table grow?

Run it

  1. 1Install uv once. It fetches Python and the lab's packages for you. This lab also runs a real system, so you need Docker.

    Terminal
    curl -LsSf https://astral.sh/uv/install.sh | sh
  2. 2Get the script. It is the whole lab: read it, then download it.

    one-old-snapshot-pins-the-version.py61 linesDownload
    View the code
    one-old-snapshot-pins-the-version.py
    # /// script
    # requires-python = ">=3.10"
    # dependencies = ["psycopg[binary]"]
    # ///
    # Copyright (c) 2026 Aayush Sharma · SPDX-License-Identifier: MIT
    # https://thedataengineering.com/lab-code/LICENSE.txt
    """One old snapshot pins the version chain.
    
    A PostgreSQL table of 1 million accounts updated at random by 8 clients for
    two minutes with autovacuum on; the table size and n_dead_tup afterwards. The
    same again while another session holds a Repeatable Read snapshot open. Then
    that session commits, VACUUM runs, and n_dead_tup and the size are read again.
    Needs the PostgreSQL service from the compose file (port 55802).
    """
    import argparse, json, random, threading, time
    import psycopg
    
    p = argparse.ArgumentParser(description=__doc__.splitlines()[0])
    p.add_argument("--dsn", default="postgresql://postgres:lab@localhost:55802/postgres", help="Connection string.")
    p.add_argument("--rows", type=int, default=1_000_000, help="Accounts.")
    p.add_argument("--seconds", type=int, default=120, help="Update time per run.")
    p.add_argument("--clients", type=int, default=8, help="Updating clients.")
    p.add_argument("--json", help="Also write the results to this JSON file.")
    args = p.parse_args()
    
    admin = psycopg.connect(args.dsn, autocommit=True)
    admin.execute("ALTER SYSTEM SET autovacuum_naptime = '5s'"); admin.execute("SELECT pg_reload_conf()")
    def reset():
        admin.execute("DROP TABLE IF EXISTS accounts")
        admin.execute(f"CREATE TABLE accounts AS SELECT g AS id, 0 AS balance, repeat('x', 80) AS filler FROM generate_series(1, {args.rows}) g")
        admin.execute("ALTER TABLE accounts ADD PRIMARY KEY (id)"); admin.execute("VACUUM ANALYZE accounts")
    def load():
        stop = time.time() + args.seconds; n = [0]
        def worker(seed):
            rng = random.Random(seed); c = psycopg.connect(args.dsn, autocommit=True)
            while time.time() < stop:
                c.execute("UPDATE accounts SET balance = balance + 1 WHERE id = %s", (rng.randrange(1, args.rows + 1),)); n[0] += 1
        ts = [threading.Thread(target=worker, args=(i,)) for i in range(args.clients)]
        for t in ts: t.start()
        for t in ts: t.join()
        return n[0]
    def stat():
        admin.execute("SELECT pg_stat_force_next_flush()"); time.sleep(1)
        size = admin.execute("SELECT pg_total_relation_size('accounts')").fetchone()[0] / 2**20
        dead = admin.execute("SELECT n_dead_tup FROM pg_stat_user_tables WHERE relname = 'accounts'").fetchone()[0]
        return size, dead
    rows = []
    reset(); s0, _ = stat(); n = load(); s, d = stat()
    rows.append({"run": "no open snapshot", "updates": n, "size_mb_before": s0, "size_mb_after": s, "n_dead_tup": d})
    reset(); s0, _ = stat()
    old = psycopg.connect(args.dsn); old.execute("BEGIN ISOLATION LEVEL REPEATABLE READ"); old.execute("SELECT count(*) FROM accounts").fetchone()
    n = load(); s, d = stat()
    rows.append({"run": "one Repeatable Read snapshot left open", "updates": n, "size_mb_before": s0, "size_mb_after": s, "n_dead_tup": d})
    old.execute("COMMIT"); old.close(); admin.execute("VACUUM accounts"); s, d = stat()
    rows.append({"run": "after the old session commits, then VACUUM", "updates": 0, "size_mb_before": None, "size_mb_after": s, "n_dead_tup": d})
    for r in rows: print(f"{r['run']:<44} updates {r['updates']:>8,}  size {r['size_mb_after']:7.1f} MB  n_dead_tup {r['n_dead_tup']:>9,}")
    
    if args.json:
        with open(args.json, "w") as f:
            json.dump({"tables": {"runs": rows}}, f)
    
  3. 3Get the services it runs against, pinned to the versions we measured. Save it next to the script.

    one-old-snapshot-pins-the-version.compose.yml13 linesDownload
    View the code
    one-old-snapshot-pins-the-version.compose.yml
    # PostgreSQL for the one-old-snapshot-pins-the-version lab.
    services:
      postgres:
        image: postgres:17.6
        command: ["postgres", "-c", "max_connections=200", "-c", "shared_buffers=256MB"]
        environment:
          POSTGRES_PASSWORD: lab
        ports: ["55802:5432"]
        healthcheck:
          test: ["CMD", "pg_isready", "-U", "postgres"]
          interval: 1s
          retries: 30
    
  4. 4Start the services, run the script with the defaults first, then change a knob from the table below and run it again. Remove the services when you are done.

    Terminal
    docker compose -f one-old-snapshot-pins-the-version.compose.yml up -d --wait
    uv run one-old-snapshot-pins-the-version.py
    uv run one-old-snapshot-pins-the-version.py --help     # every knob
    docker compose -f one-old-snapshot-pins-the-version.compose.yml down -v
Knobs
Knob · defaultWhat it changes
--dsnpostgresql://postgres:lab@localhost:55802/postgres Connection string.
--rows1,000,000 Accounts.
--seconds120 Update time per run.
--clients8 Updating clients.

Our run

Two minutes of updates

Two minutes of updates
RunUpdatesSize (MB)n_dead_tup
no open snapshot1.02M14637.2K
one Repeatable Read snapshot left open1.08M2941.08M
after the old session commits, then VACUUM02940
Reference runApple M2 · 8 cores · 8 GB
Output · uv run one-old-snapshot-pins-the-version.py
no open snapshot                             updates 1,018,984  size   145.8 MB  n_dead_tup    37,208
one Repeatable Read snapshot left open       updates 1,084,803  size   293.6 MB  n_dead_tup 1,079,667
after the old session commits, then VACUUM   updates        0  size   293.6 MB  n_dead_tup         0
macOS 26.6.2 · Python 3.13.3 · psycopg 3.3.6 · postgres:17.6 · Docker 28.5.1 · run 2026-10-20

What we found

Without an open snapshot, 1.02M updates left the table at 146 MB with 37.2K dead tuples: autovacuum kept up. With one Repeatable Read snapshot open, a similar number of updates left 1.08M dead tuples, essentially every old version, and the table grew to 294 MB. After the session committed and VACUUM ran, dead tuples fell to 0, and the size stayed at 294 MB.


Read it

The oldest snapshot sets the cleanup horizon for everyone, so the cost of a long transaction is paid by every writer in the database. Monitor the age of the oldest open snapshot, time out idle transactions, and run long reports on a replica or from a snapshot export, not inside a transaction on the primary.