Journaling: the filesystem's own WAL
What does a journaling filesystem not promise?
It is the same mechanism, one layer lower, with a deliberately smaller scope. The scope choice is the interesting part.
A journaling filesystem (ext4, XFS, NTFS and others) writes a log of its own bookkeeping changes before making them. By default it journals metadata, not file data. Metadata means the filesystem's internal bookkeeping: which blocks are free (allocation bitmaps), which blocks belong to which file (inode pointers), directory entries.
Why only metadata? Because of blast radius (how much damage one error causes):
- Corrupt metadata can lose huge amounts of other people's data and can make the whole volume unmountable.
- Corrupt data loses only one file's contents.
Metadata is small and catastrophic. Data is large and locally damaging. So the filesystem pays the 2× write only on the small, catastrophic half. This is correct expected-loss reasoning, not a weakness. But it has a consequence that surprises many engineers:
Many engineers believe the second promise. The ext4 modes make the difference explicit:
| Mode | What it does | Consequence |
|---|---|---|
data=journal | data also goes through the journal | true atomicity, 2× write for everything |
data=ordered (default) | data blocks are forced before the metadata commit that points to them | you never see someone else's old deleted blocks in your new file; your file may still be incomplete |
data=writeback | no ordering | after a crash a file may legally contain stale blocks of another user's deleted data. This is a confidentiality bug disguised as a performance option |
Two consequences worth remembering
Journals group commit by design. ext3 and ext4 collect all activity into compound transactions on an interval of about 5 seconds. This produced a classic problem on ext3: one process called fsync() on a 4 KB file and had to wait for gigabytes of another process's dirty data, because both belonged to the same compound transaction. A batch is a shared fate boundary. Shared fate converts other people's variance into your latency. (Predictable cost is better than occasional huge cost: variance hurts more than volume.)
Logs stack, and stacked logs interfere. A database WAL on a journaling filesystem on top of an SSD's flash translation layer (the SSD's own internal remapping and GC) is three append-only logs, and stacked logs multiply write amplification instead of cancelling.
The most common durability bug in application code
On ext4 with delayed allocation (the filesystem postpones choosing disk blocks until it writes back), the safe way to replace a file is:
// 1. write the new content to a temporary file in the SAME directory
int fd = open("settings.tmp", O_WRONLY | O_CREAT | O_TRUNC, 0644);
write(fd, data, len);
fsync(fd); // 2. make the temp file's content durable
close(fd);
rename("settings.tmp", "settings.json"); // 3. atomically replace the old file
int dirfd = open(".", O_RDONLY | O_DIRECTORY);
fsync(dirfd); // 4. make the rename itself durable
close(dirfd);The last step, fsync on the directory, is the one almost everyone leaves out. Without it, the rename may not be durable even though the file is. Because so much software got this wrong and left zero-length files after crashes, ext4 added auto_da_alloc heuristics in 2009. That is a case of a layer adding a compatibility hack because its true contract was stricter than the idiom everyone had adopted.