Two liabilities: recovery and space

Which two liabilities does every commit build up, and what does recovery time really depend on?

Logs and undo and Group commit and fsync bought cheap commits by deferring work. The log lets you say "yes" after one sequential force and settle up later. This article is the bill arriving.

Two different liabilities build up on every commit, and most explanations mix them up:

• A recovery liability: the distance back to a point where you know how the on-disk state relates to the log. Measured in time to restart.
• A space liability: how much log you must physically keep. Measured in bytes retained.

They have different collectors and different release conditions. The central mistake is to think one pays off the other:

The guiding question: since the log is an unbounded liability that grows at the write rate, what is the cheapest way to create a recent, trustworthy starting point, and who must agree before the past may be thrown away?

The log is the only part of the system that grows in proportion to your success. Every mechanism here is a negotiation with someone who has not finished reading it yet.

The equation this article optimizes

Recovery must replay the log from the oldest record whose effect might not be on disk. So:

RTO ≈ (checkpoint_interval × log_write_rate) / replay_rate

The term that people get wrong is replay_rate. Writing the log is sequential. Redo is random reads, because each record must first fetch its page before it can be applied. On a cold buffer pool (memory just emptied by the restart), replay is limited by random page reads and can run slower than the workload that produced the log. This is the mechanism behind the old InnoDB advice that a large redo log means unusable recovery.

This asymmetry is also why ARIES recovery has three passes, not two:

1. Analysis: read the log from the last checkpoint and rebuild the table of dirty pages (pages that were changed in memory but may not be on disk) and the list of transactions that were active.
2. Redo: "repeat history": re-apply every logged change that the pages do not yet contain, for all transactions, even those that will be rolled back.
3. Undo: roll back the transactions that had not committed ("the losers").

The analysis pass exists so that the redo pass knows which pages it will need before it starts, and can prefetch them in batches, turning random reads into something organized. Recovery speed is bought with a pass that does no recovery. During undo, each reversal is itself logged (a compensation log record, CLR) so that a crash during recovery does not undo the same change twice, and the work stays bounded.

Why redo the losers at all, if undo is about to reverse them? Undo operations, especially "physiological" ones on index pages, are defined relative to the exact state at the moment of the crash. Restoring that state first makes undo a well-defined inverse. You cannot skip redo selectively, because at redo time you do not yet know what undo will need to see. And because each CLR carries an undoNextLSN (the next record still to undo) and is never itself undone, a crash in the middle of a long rollback resumes where it stopped instead of starting over. Crashes during recovery are the normal case, not the edge case, and ARIES is the standard demonstration of designing for that.

One practical correction to the equation above: ARIES checkpoints are fuzzy, so the real bound on redo distance is the oldest dirty page, set by how far behind the background page cleaner is, not by the checkpoint interval alone. This is why tuning the background writer can move recovery time more than tuning checkpoint timeouts, and why a system under heavy write load recovers more slowly than the same system idle.

Write the equation as a ratio: recovery costs write_rate / replay_rate seconds of downtime for each second of checkpoint interval.