Reader-writer lock against plain mutex on a read-heavy workload
Measure your machineNeeds C compiler, Python
Problem
A reader-writer lock lets any number of readers in at once, so it looks like the answer to a read-mostly workload. But taking a read lock means incrementing a shared reader count, and releasing it means decrementing it: every read writes the same cache line. Readers that write do not scale, whatever the lock's semantics allow. An immutable snapshot published through one atomic pointer lets readers load the pointer and read, writing nothing shared; writers copy, modify and swap.
Make a shared counter table with 1,000 entries. Start N threads (1, 2, 4, 8, 16) where 99% of operations are reads and 1% are writes. Run it with (a) a plain mutex, (b) a reader-writer lock (pthread_rwlock_t), (c) an immutable snapshot read through one atomic pointer. Plot total operations per second against N. The script compiles and runs the C program for you; this machine has 8 cores.
The question. Does the reader-writer lock scale better than a mutex on a 99%-read workload, and does the snapshot pointer scale with readers?
Predict
Expect (b) to scale poorly or even fall as N grows, because every read lock writes to a shared counter, and expect (c) to scale almost linearly for readers.
Run it
1Install uv once. It fetches Python and the lab's packages for you.
Terminal curl -LsSf https://astral.sh/uv/install.sh | sh2Get the script. It is the whole lab: read it, then download it.
View the codeHide the code
reader-writer-lock-against-plain-mutex.py # /// script # requires-python = ">=3.10" # /// # Copyright (c) 2026 Aayush Sharma · SPDX-License-Identifier: MIT # https://thedataengineering.com/lab-code/LICENSE.txt """Reader-writer lock against a plain mutex (and a snapshot pointer). A shared table of 1,000 counters; N threads doing 99% reads and 1% writes for a fixed time, protected by a mutex, by a pthread reader-writer lock, and by an immutable snapshot read through one atomic pointer (writers copy and swap). Total operations per second against N. The C program is compiled and run by this script (needs cc). """ import argparse, json, os, subprocess, tempfile p = argparse.ArgumentParser(description=__doc__.splitlines()[0]) p.add_argument("--threads", type=lambda s: [int(v) for v in s.split(",")], default=[1, 2, 4, 8, 16], help="Thread counts N.") p.add_argument("--seconds", type=float, default=1.0, help="Seconds per measurement.") p.add_argument("--json", help="Also write the results to this JSON file.") args = p.parse_args() C = r""" #include <pthread.h> #include <stdatomic.h> #include <stdio.h> #include <stdlib.h> #include <string.h> #include <time.h> #define E 1000 #define RING 64 long table[E]; pthread_mutex_t mu = PTHREAD_MUTEX_INITIALIZER; pthread_rwlock_t rw = PTHREAD_RWLOCK_INITIALIZER; long *snaps[RING]; _Atomic(long *) current; _Atomic long next_snap = 1; pthread_mutex_t wmu = PTHREAD_MUTEX_INITIALIZER; volatile int stop = 0; static double now(void) { struct timespec t; clock_gettime(CLOCK_MONOTONIC, &t); return t.tv_sec + t.tv_nsec / 1e9; } static void *worker(void *arg) { unsigned s = (unsigned)(long)arg * 2654435761u; long ops = 0, acc = 0; while (!stop) { s = s * 1103515245 + 12345; int i = (s >> 8) % E; int write = (s >> 4) % 100 == 0; #if MODE == 0 pthread_mutex_lock(&mu); if (write) table[i]++; else acc += table[i]; pthread_mutex_unlock(&mu); #elif MODE == 1 if (write) { pthread_rwlock_wrlock(&rw); table[i]++; } else { pthread_rwlock_rdlock(&rw); acc += table[i]; } pthread_rwlock_unlock(&rw); #else if (write) { /* copy, modify, publish (old copies are reused from a ring) */ pthread_mutex_lock(&wmu); long *cur = atomic_load(¤t); long *nx = snaps[atomic_fetch_add(&next_snap, 1) % RING]; memcpy(nx, cur, sizeof(long) * E); nx[i]++; atomic_store(¤t, nx); pthread_mutex_unlock(&wmu); } else { acc += atomic_load_explicit(¤t, memory_order_acquire)[i]; } #endif ops++; } return (void *)(ops + (acc & 0)); } int main(int argc, char **argv) { int n = atoi(argv[1]); double secs = atof(argv[2]); for (int r = 0; r < RING; r++) snaps[r] = calloc(E, sizeof(long)); atomic_store(¤t, snaps[0]); pthread_t th[64]; for (long i = 0; i < n; i++) pthread_create(&th[i], NULL, worker, (void *)(i + 1)); struct timespec ts = { (time_t)secs, (long)((secs - (time_t)secs) * 1e9) }; nanosleep(&ts, NULL); stop = 1; /* the main thread sleeps: it must not take a core */ long total = 0; for (int i = 0; i < n; i++) { void *r; pthread_join(th[i], &r); total += (long)r; } printf("%.0f\n", total / secs); } """ d = tempfile.mkdtemp(); src = os.path.join(d, "rw.c"); open(src, "w").write(C) rows = [] for mode, label in ((0, "mutex"), (1, "reader-writer lock"), (2, "snapshot pointer")): exe = os.path.join(d, f"rw{mode}") subprocess.run(["cc", "-O2", "-pthread", src, "-o", exe, f"-DMODE={mode}"], check=True) for n in args.threads: ops = float(subprocess.run([exe, str(n), str(args.seconds)], capture_output=True, text=True, check=True).stdout) rows.append({"lock": label, "threads": n, "mops": ops / 1e6}) print(f"{label:<20} " + " ".join(f"N={r['threads']}: {r['mops']:6.1f}" for r in rows if r["lock"] == label) + " million ops/s") if args.json: with open(args.json, "w") as f: json.dump({"tables": {"throughput": rows}}, f)3Run it in the folder you saved it to, with the defaults first. Then change a knob from the table below and run it again.
Terminal uv run reader-writer-lock-against-plain-mutex.py uv run reader-writer-lock-against-plain-mutex.py --help # every knob
| Knob · default | What it changes |
|---|---|
| --threads1, 2, 4, 8, 16 | Thread counts N. |
| --seconds1 | Seconds per measurement. |
Our run
Data table
| Threads | series | Million ops/s |
|---|---|---|
| 1 | mutex | 158 |
| 2 | mutex | 85.6 |
| 4 | mutex | 25.3 |
| 8 | mutex | 42.3 |
| 16 | mutex | 44.9 |
| 1 | reader-writer lock | 126 |
| 2 | reader-writer lock | 35.8 |
| 4 | reader-writer lock | 9.37 |
| 8 | reader-writer lock | 3.87 |
| 16 | reader-writer lock | 2.7 |
| 1 | snapshot pointer | 386 |
| 2 | snapshot pointer | 223 |
| 4 | snapshot pointer | 145 |
| 8 | snapshot pointer | 109 |
| 16 | snapshot pointer | 115 |
mutex N=1: 158.1 N=2: 85.6 N=4: 25.3 N=8: 42.3 N=16: 44.9 million ops/s
reader-writer lock N=1: 125.5 N=2: 35.8 N=4: 9.4 N=8: 3.9 N=16: 2.7 million ops/s
snapshot pointer N=1: 385.6 N=2: 223.3 N=4: 145.5 N=8: 108.7 N=16: 115.4 million ops/sWhat we found
The reader-writer lock fell the hardest of the three, from 126 million operations per second on one thread to 2.7 million on 16, below the plain mutex at every N above one (the mutex settled around 44.9 million). The snapshot pointer was fastest at every N but did not scale either: 386 million on one thread, 109 million on eight. Its readers write nothing shared, but the 1% of writers swap the pointer, and every swap invalidates the line every reader loads, so at these rates the writers' traffic sets the ceiling.
Read it
The claim held in its strong form: a reader that writes a shared counter does not scale, and a reader-writer lock made a read-mostly workload slower than a mutex. The snapshot pointer removed the readers' writes and was several times faster, but "almost linear" assumes writes are rare enough that the pointer's line stays in every reader's cache. With 1% writes at hundreds of millions of operations per second, it did not.
Scaling is a property of what crosses cores per operation. Count the shared writes per operation, including the ones the lock does for you.