Skip to content
About
Main content

Phantoms and gap locks depend on the index

Run a real systemNeeds Docker, Python

Problem

InnoDB's REPEATABLE READ prevents in locking reads by locking not just the rows a range query returns but the gaps between index entries, so no new row can appear in the range. What gets locked depends on the plan: with an index on the filtered column, the cover the range in that index; without one, the query scans the whole table, and every row and gap it passes is locked, so inserts anywhere wait.

On MySQL/InnoDB (default REPEATABLE READ), create seats(id int primary key, section char(1), price int) with and without INDEX(price). In session 1 run BEGIN; SELECT * FROM seats WHERE price BETWEEN 100 AND 120 FOR UPDATE;. In session 2 insert a seat priced inside the range and one priced far outside it. Look at performance_schema.data_locks.

The question. With and without an index on price, which inserts wait, and how many record locks does the range query hold?


Predict

Without an index on price, does an insert at price 900, far outside the queried range, wait?

Run it

  1. 1Install uv once. It fetches Python and the lab's packages for you. This lab also runs a real system, so you need Docker.

    Terminal
    curl -LsSf https://astral.sh/uv/install.sh | sh
  2. 2Get the script. It is the whole lab: read it, then download it.

    phantoms-and-gap-locks-depend-on.py48 linesDownload
    View the code
    phantoms-and-gap-locks-depend-on.py
    # /// script
    # requires-python = ">=3.10"
    # dependencies = ["pymysql", "cryptography"]
    # ///
    # Copyright (c) 2026 Aayush Sharma · SPDX-License-Identifier: MIT
    # https://thedataengineering.com/lab-code/LICENSE.txt
    """Phantoms and gap locks depend on the plan.
    
    MySQL/InnoDB at its default REPEATABLE READ. Session 1 runs SELECT ... WHERE
    price BETWEEN 100 AND 120 FOR UPDATE and holds it; session 2 tries to insert
    a seat priced inside the range and one priced far outside it. With an index on
    price, then without, records which inserts wait (lock wait timeout of 2 s) and
    counts the locks in performance_schema.data_locks.
    Needs the MySQL service from the compose file (port 55810).
    """
    import argparse, json, time
    import pymysql
    
    p = argparse.ArgumentParser(description=__doc__.splitlines()[0])
    p.add_argument("--port", type=int, default=55810, help="MySQL port.")
    p.add_argument("--rows", type=int, default=10_000, help="Seats.")
    p.add_argument("--json", help="Also write the results to this JSON file.")
    args = p.parse_args()
    
    def conn(): return pymysql.connect(host="127.0.0.1", port=args.port, user="root", password="lab", database="lab", autocommit=False)
    admin = conn(); cur = admin.cursor()
    rows = []
    for indexed in (True, False):
        cur.execute("DROP TABLE IF EXISTS seats")
        cur.execute("CREATE TABLE seats (id int PRIMARY KEY, section char(1), price int" + (", INDEX (price)" if indexed else "") + ") ENGINE=InnoDB")
        cur.executemany("INSERT INTO seats VALUES (%s, %s, %s)", [(i, "ABCDE"[i % 5], (i * 37) % 1000) for i in range(1, args.rows + 1)]); admin.commit()
        s1 = conn(); c1 = s1.cursor(); c1.execute("BEGIN"); c1.execute("SELECT * FROM seats WHERE price BETWEEN 100 AND 120 FOR UPDATE"); matched = len(c1.fetchall())
        cur.execute("SELECT lock_type, lock_mode, count(*) FROM performance_schema.data_locks GROUP BY lock_type, lock_mode"); locks = cur.fetchall(); admin.commit()
        n_record = sum(n for t, m, n in locks if t == "RECORD")
        for label, price, sid in (("insert at price 110 (inside the range)", 110, 99_001), ("insert at price 900 (far outside)", 900, 99_002)):
            s2 = conn(); c2 = s2.cursor(); c2.execute("SET innodb_lock_wait_timeout = 2"); t = time.perf_counter()
            try: c2.execute("INSERT INTO seats VALUES (%s, 'F', %s)", (sid, price)); outcome = "inserted"; s2.rollback()
            except pymysql.err.OperationalError as e: outcome = "waited, then lock wait timeout" if e.args[0] == 1205 else str(e)
            rows.append({"index_on_price": "yes" if indexed else "no", "insert": label, "outcome": outcome, "waited_s": time.perf_counter() - t,
                         "record_locks": n_record, "rows_matched": matched})
            s2.close()
        s1.rollback(); s1.close()
    for r in rows: print(f"index on price: {r['index_on_price']:<3}  {r['insert']:<40} {r['outcome']:<32} ({r['waited_s']:.1f} s)  record locks held: {r['record_locks']:,}")
    
    if args.json:
        with open(args.json, "w") as f:
            json.dump({"tables": {"inserts": rows}}, f)
    
  3. 3Get the services it runs against, pinned to the versions we measured. Save it next to the script.

    phantoms-and-gap-locks-depend-on.compose.yml13 linesDownload
    View the code
    phantoms-and-gap-locks-depend-on.compose.yml
    # MySQL (InnoDB) for the gap-lock lab.
    services:
      mysql:
        image: mysql:8.4
        environment:
          MYSQL_ROOT_PASSWORD: lab
          MYSQL_DATABASE: lab
        ports: ["55810:3306"]
        healthcheck:
          test: ["CMD", "mysqladmin", "ping", "-h", "127.0.0.1", "-uroot", "-plab"]
          interval: 2s
          retries: 60
    
  4. 4Start the services, run the script with the defaults first, then change a knob from the table below and run it again. Remove the services when you are done.

    Terminal
    docker compose -f phantoms-and-gap-locks-depend-on.compose.yml up -d --wait
    uv run phantoms-and-gap-locks-depend-on.py
    uv run phantoms-and-gap-locks-depend-on.py --help     # every knob
    docker compose -f phantoms-and-gap-locks-depend-on.compose.yml down -v
Knobs
Knob · defaultWhat it changes
--port55,810 MySQL port.
--rows10,000 Seats.

Our run

Inserts while the range is locked

Inserts while the range is locked
Index on priceInsertOutcomeRecord locks held
yesinsert at price 110 (inside the range)waited, then lock wait timeout421
yesinsert at price 900 (far outside)inserted421
noinsert at price 110 (inside the range)waited, then lock wait timeout10K
noinsert at price 900 (far outside)waited, then lock wait timeout10K
Reference runApple M2 · 8 cores · 8 GB
Output · uv run phantoms-and-gap-locks-depend-on.py
index on price: yes  insert at price 110 (inside the range)   waited, then lock wait timeout   (2.0 s)  record locks held: 421
index on price: yes  insert at price 900 (far outside)        inserted                         (0.0 s)  record locks held: 421
index on price: no   insert at price 110 (inside the range)   waited, then lock wait timeout   (2.0 s)  record locks held: 10,020
index on price: no   insert at price 900 (far outside)        waited, then lock wait timeout   (2.0 s)  record locks held: 10,020
macOS 26.6.2 · Python 3.13.3 · pymysql 1.2.3 · cryptography 50.0.2 · mysql:8.4 · Docker 28.5.1 · run 2026-10-20

What we found

With the index, the range query held 421 record locks; the insert inside the range waited, then lock wait timeout, and the one far outside was inserted. Without the index, the same query held 10K locks, every row of the table and its gaps, and even the insert at price 900 waited, then lock wait timeout.


Read it

The isolation level names a guarantee; the plan decides what it costs. A locking read without a usable index locks the table in effect, so a missing index turns into a concurrency outage, not just a slow query. Check the plan of every locking read.