Resharding: the map is harder than the bytes

Changing P is where the design bites. The reason is not the data movement. It is that the partition map is a piece of mutable state cached by every client, every router and every peer, with no mechanism that can force a refresh. It is a cache you cannot invalidate, now holding a fact critical for correctness rather than for performance.

The universal answer is not to try. Clients act on a stale map, and the destination node detects the staleness and corrects it:

• Redis Cluster answers MOVED;
• Kafka answers NOT_LEADER_FOR_PARTITION;
• TiKV reports a region epoch mismatch;
• HBase throws a region-moved exception.

In every case the response carries the new mapping, so the error path is also the refresh path. The cost lands unevenly: active clients are corrected at once, and the least active client discovers the change at the worst moment.

The map carries a version, and the version is the fencing token. A request tagged with an old epoch is rejected on arrival. This is the fencing built for leaders, applied to ownership instead, and the shard-map epoch is the ownership map's designated version cell.

The consequence for design is that most systems refuse to change P at all. A fixed partition count makes rebalancing a pure assignment change: move whole partitions, never split them. The content of the map changes while its shape does not.

The standard form maps keys to a large, fixed number of logical shards (say 1,024; they are also called vnodes, slots or buckets). Then it maps logical shards to physical nodes. Rebalancing moves logical shards between nodes. The mapping from key to logical shard never changes.

key  --hash-->  logical shard (0..1023)  --lookup table-->  physical node
        never changes                          changes when you add or remove nodes

This separates two things that were mixed up: the hash space is stable, and the placement is flexible. Examples: Redis Cluster's 16,384 slots, Elasticsearch's fixed shard count per index, Cassandra's vnodes. Pick the logical count generously at the start. It is the one number you cannot change later. The usual advice is a few hundred to a few thousand: enough for many years of growth, few enough that the per-shard overhead stays small.

The cost is "a fixed number meets growth" in its plainest form: you chose a number against future growth. A log's partition count is the same bet, and changing it breaks per-key order (changing N later). Systems that split dynamically pay instead with an atomic ownership handover per split, which is a consensus operation. That is why the systems that do it are the ones that already had per-partition consensus groups.

Compression: resharding is a distributed cache-invalidation problem wearing a data-movement costume; the map epoch is the fence, and a fixed partition count is the usual way of refusing the problem.