A backend process reads a configuration file for the second time. The first read required storage I/O, but the second finishes much faster even though the application did not create its own cache.
The same process writes a large log record. write() returns quickly, but the storage device remains busy afterward.
Both observations come from the page cache: RAM that the operating system uses to cache file contents. Reads can be served from cached memory, and ordinary buffered writes can update cached memory before the changed data is written back to storage.
The page cache makes file I/O dramatically faster, but it also separates application-visible operations from device activity:
A file operation can finish using RAM while storage I/O happens earlier, later, or not at all.
To reason about file performance, memory usage, and write stalls, we need to follow cached pages through their clean, dirty, and write-back states.
Main memory is much faster than persistent storage. If the kernel discarded file data immediately after every read, repeated access would repeatedly pay storage latency.
Instead, the kernel keeps recently used file contents in RAM:
The second read can avoid a device request if the required range is still cached.
The cache also helps writes. Copying changed bytes into RAM is usually much faster than waiting for a storage device. The kernel can accept several writes, organize them, and send larger or better-grouped operations to the storage stack later.
This provides three major benefits:
The page cache is not a fixed-size reserved area. Linux uses otherwise available memory and reclaims cache pages when applications or the kernel need that memory for something more valuable.
The page cache holds file contents in page-sized regions of memory.
Conceptually, a cached region is identified by:
It is not identified by a pathname. Two pathnames can refer to the same underlying file, and a file can be renamed while open. The cache belongs to the kernel's file object and its offset-indexed contents, not to the spelling of the name used to open it.
Suppose a process reads bytes 8,192 through 12,287 of a file. The kernel can look up the cache entry covering that file range. Another process reading the same underlying file range can use the same cached data.
This makes the page cache a system-wide resource:
One copy in memory serves all three. A second process reading the same file usually pays no storage cost at all.
The processes still have separate address spaces and user buffers. What they share indirectly is the kernel's cached copy of the file contents.
The traditional name is page cache, and cached file data is commonly explained in memory pages. Modern Linux frequently manages page-cache memory with a structure called a folio, which can represent one or more base pages.
The distinction matters when reading kernel code, but the operating-system model remains the same: the cache tracks file-offset ranges in RAM along with state such as whether their contents are current, dirty, or under write-back.
Executable code, shared libraries, and memory-mapped files can all use file-backed cached pages. The page cache is therefore part of ordinary program execution, not only explicit read() and write() calls.
It is different from:
All of these can hold copies of related data, but they operate at different layers and provide different guarantees.
A cached file region can move through several important states.
Clean means the cached contents agree with the backing file state known to the kernel. If the page is no longer useful, it can usually be discarded because the data can be read again from storage.
Dirty means the cached contents have been modified and are newer than the backing storage. The page cannot simply be discarded; doing so would lose the changes.
Writeback means the kernel has started sending the changed contents toward the backing storage and is waiting for that write-back operation to finish.
A page can be modified again while write-back is in progress. The kernel must preserve that newer change rather than marking the page permanently clean when the older write completes.
These states explain a key difference under memory pressure:
Consider:
Assume fd refers to an ordinary file opened for buffered I/O.
The kernel identifies the file and offset range requested by the open file description. It then looks for the corresponding data in the page cache.
If the required range is present and up to date, the read is a page-cache hit:
The application calls read(), the kernel finds the file range already in the page cache, the bytes are copied into the application's buffer, and read() returns.
No storage-device command is needed for those bytes. The operation still crosses the system-call boundary and copies or maps data as required, so it is not free, but it avoids the much larger cost of a storage access.
If the range is absent, the read is a page-cache miss.
The kernel must:
read().The cache is populated on the way back, so the next reader of this range takes the short path instead.
The newly loaded contents remain in the page cache after the call. A later read can hit them unless memory pressure or file changes cause them to be reclaimed or invalidated.
Sequential access often continues into nearby file regions. After detecting that pattern, Linux can request pages beyond the one currently needed.
If an application reads page 20, then 21, then 22, the kernel may fetch later pages before the application asks for them:
Useful read-ahead overlaps storage I/O with application work and converts future misses into hits. Incorrect read-ahead wastes I/O and cache space, so the kernel adapts rather than fetching an entire file unconditionally.
Random access provides less reliable evidence about what will be needed next and benefits less from read-ahead.
File-backed memory mappings do not create an unrelated second cache of file contents.
When a process first accesses a mapped file region that is not resident, the resulting fault can load that region through the page cache. A later read() of the same file range can observe the cached contents, and a different process mapping the same file can share the same underlying cache page.
For a shared writable mapping, modifying the mapped memory makes the corresponding file-backed cache region dirty. It can then follow the same write-back path as data changed through buffered write().
A private writable mapping behaves differently: modifications use private copy-on-write memory rather than changing the underlying file. The initial file contents can still come from the page cache.
This unified cache keeps ordinary buffered I/O and file-backed mappings coherent at the kernel level.
Now consider:
For an ordinary buffered file write, the foreground path commonly looks like:
The call returns before the storage write happens. A successful write() therefore says the kernel accepted the bytes, not that they survived a power loss.
The process can reuse its original buffer after the kernel accepts the bytes because the kernel now has its own cached copy.
The storage device may not have received a command when write() returns. Success means the kernel accepted the bytes according to the file operation's semantics; it does not by itself establish that the new contents would survive a power loss.
That separation is the source of both speed and risk. Foreground writes can run at memory speed for a while, but dirty memory is unfinished storage work that the kernel must eventually process.
If an application changes only part of a cached page, the unchanged bytes must remain correct.
When the page is already cached, the kernel can update only the requested portion. If the page is absent and existing contents must be preserved, the kernel may first need to obtain them before applying the partial change.
This is one reason a small application write does not map mechanically to one equally sized device command.
Write-back is the process of sending dirty file-cache contents to their backing storage.
A simplified lifecycle is:
The filesystem converts dirty file ranges into storage mappings, and the block layer prepares requests for the device driver. Nearby dirty regions can often be written together.
Write-back happens asynchronously in many ordinary cases. Kernel flusher work can process dirty pages while the application continues running.
Suppose an application can copy data into memory at 5 GB/s, while the storage device can sustain only 500 MB/s.
For a short burst, buffered writes may appear to run near the memory-copy rate. Dirty memory grows because the application produces changes faster than storage can absorb them.
This cannot continue indefinitely. Once dirty memory approaches its allowed range, the kernel slows the processes creating it. Long-running throughput eventually has to approach what the backing storage can sustain.
This explains a common benchmark shape:
A fast initial burst lets dirty memory accumulate. Write-back then catches up, writers are throttled, and sustained throughput settles lower than the burst suggested.
The initial number measures buffering capacity as much as storage performance.
Linux does not wait for dirty memory to consume all RAM. Several mechanisms initiate or accelerate write-back.
When dirty memory reaches a background threshold, kernel flusher work begins writing dirty data in the background.
The writer can often continue while write-back attempts to keep dirty growth under control.
At a higher dirty threshold, a process generating writes must participate in restoring balance. The kernel can throttle it and make it perform or wait for write-back work.
This applies backpressure:
When a writer produces dirty pages too quickly and the dirty threshold is approached, the writer is slowed so storage gets time to catch up.
The threshold is not extra storage capacity. It is a control point that prevents unbounded dirty-memory growth.
Dirty data does not have to reach a global threshold before being considered for write-back.
Once data has remained dirty long enough, it becomes eligible for periodic flusher work. This prevents a small amount of dirty data from remaining in memory indefinitely merely because the system never reaches a size threshold.
The memory-management subsystem may need to reclaim cache pages. Clean pages can often be dropped immediately, but dirty pages must first begin write-back.
Memory pressure can therefore increase write-back activity even when the workload did not explicitly request it.
Kernel subsystems and applications can request that particular dirty data be written out rather than waiting only for background policy.
The existence of such a request does not change the core state transition: dirty cache must be submitted, complete successfully, and become clean.
Loading simulation...
Linux exposes system-wide write-back controls under /proc/sys/vm.
| Control | Meaning |
|---|---|
dirty_background_ratio | Percentage at which background flusher work starts |
dirty_background_bytes | Byte-based alternative to the background ratio |
dirty_ratio | Percentage at which a process generating writes starts write-back and is throttled |
dirty_bytes | Byte-based alternative to the foreground ratio |
dirty_expire_centisecs | Age after which dirty data is eligible for periodic write-back |
dirty_writeback_centisecs | Interval between periodic flusher wakeups |
The ratio and byte forms are paired alternatives:
Only one form in each pair is active. Writing one causes its counterpart to appear as zero.
The ratios are based on a kernel calculation of available memory containing free and reclaimable pages. They are not simple percentages of MemTotal, and they should not be calculated directly from the MemAvailable line as though the two definitions were identical.
The centisecs suffix means hundredths of a second:
These controls describe policy, not exact promises that every page will be submitted at one precise byte count or age. Linux also balances dirtying against the capabilities of individual backing devices.
Inspect the current values without changing them:
Defaults and appropriate settings vary. Changing them without measuring the workload can replace frequent small write-back with large stalls, or reduce useful buffering without improving application latency.
Linux does not try to keep MemFree as large as possible. Completely unused RAM provides no performance benefit, while clean cached file data can avoid future I/O.
When memory is needed, the kernel chooses among reclaimable candidates.
A clean file-cache page has a valid copy on backing storage. If it is not actively useful and has no condition preventing reclaim, the kernel can discard it and reuse the RAM.
If the file data is needed later, it can be read again.
A dirty page is the only current copy of changed file contents. The kernel cannot discard it safely.
It must:
Write-back pages are also tied to in-flight I/O and cannot simply be reused until the relevant operation completes.
On a slow or overloaded device, reclaim can therefore stall behind storage. A system can have plenty of nominally reclaimable file memory yet struggle to reclaim it quickly because much of it is dirty or under write-back.
Linux tracks access patterns so frequently or recently used pages have a better chance of remaining cached than cold pages.
A large one-time scan can still displace useful cached data. After the scan, a backend service may experience a burst of cache misses while its working set becomes resident again.
This is why warm-cache and cold-cache measurements can differ sharply even when the application and device are unchanged.
/proc/meminfo exposes several useful counters:
Representative output:
The fields mean:
Cached includes in-memory file cache and also memory such as tmpfs and shared memory. It is not a pure count of cached disk-file bytes.Dirty is memory waiting to be written back.Writeback is memory actively being written back.MemAvailable estimates how much memory can be made available for new work without swapping.Some /proc/meminfo counters overlap, so they should not all be added together.
The free command presents a higher-level view:
The buff/cache column being large is normally healthy. The available column is usually more useful than free when deciding whether memory is genuinely scarce.
In one terminal, sample the counters:
In a disposable directory, create a moderately sized file with ordinary buffered writes:
Depending on RAM size and storage speed, Dirty may rise while the command runs and fall as write-back catches up. Writeback can be brief enough to miss between samples on a fast device.
Remove the demonstration file afterward:
Avoid treating /proc/sys/vm/drop_caches as a routine cache-management tool. Linux reclaims cache automatically, and forcing useful cache out can create significant CPU and I/O work when applications need the data again.
The page cache often explains performance that otherwise appears inconsistent.
A repeated file read may run at memory speed because the data is cached. A benchmark that intends to measure a storage device but repeatedly reads a small file may mostly measure memory copying and system-call overhead.
Both warm-cache and cold-cache behavior can matter. A long-running service benefits from a warm working set, while startup after reboot or cache eviction exposes storage performance.
A logging process can write faster than the device for a short period by accumulating dirty pages. If the producer stays faster than write-back, dirty throttling eventually appears as application latency.
The stall may seem sudden because the early calls were absorbed by RAM. The device did not become slow at the threshold; the earlier speed was not sustainable.
A backup, file scan, or large build can fill the page cache with one-time data. Even if that job uses mostly sequential I/O, it can displace hot file pages used by a latency-sensitive service.
Afterward, the service pays storage latency to rebuild its cache residency.
A database may keep frequently used pages in its own buffer pool while the same file data also resides in the kernel page cache.
That duplication can be useful when the kernel cache serves other readers, but it can also consume memory without increasing the database's effective working set. The impact depends on the database, access path, and memory pressure.
One process can generate enough dirty data to cause write-back and throttling that affects other workloads using the same backing device or competing for memory.
Per-process write speed should therefore be interpreted alongside system-wide Dirty, Writeback, device throughput, and application latency.
The page cache stores file contents in RAM by file identity and offset. Cached reads avoid storage I/O, read-ahead anticipates sequential access, and file-backed mappings use the same underlying cache. Clean pages can usually be reclaimed because storage already contains their data.
Buffered writes copy changed bytes into the page cache and mark the affected regions dirty. Write-back later submits those changes through the filesystem and block layer. Dirty and write-back pages cannot be discarded like ordinary clean cache, and sustained writers are throttled when storage cannot keep up.
Linux controls dirty memory with background and foreground thresholds, age-based eligibility, periodic flusher work, and memory-pressure reclaim. /proc/meminfo exposes Cached, Dirty, and Writeback, while MemAvailable helps distinguish useful cache occupancy from genuine memory scarcity.
5 quizzes