Durability and fsync
Suppose an order service writes a payment record to a file. write() returns successfully, so the service tells the client the payment is committed.
A few milliseconds later, the machine loses power. After reboot, the record is gone.
Nothing necessarily went wrong. With buffered I/O, write() can return once the kernel has accepted the data into memory. The data may not have reached persistent storage yet.
For applications that require durability, these are different events:
- The kernel accepted the data.
- Other code can see the data.
- Writes occurred in the required order.
- The data will survive a crash or power loss.
A successful
write()does not necessarily mean the data is durable.
Operations such as fsync() exist to bridge the gap between data accepted by the kernel and data safely persisted to storage.
This chapter separates acceptance, visibility, ordering, and durability, follows data through volatile caches, and explains fsync, fdatasync, directory synchronization, durable replacement, per-write options, and group commit.