A cache works by keeping useful data in memory, close to the application. But memory is limited. Sooner or later, the cache has to make room for new data.

Cache eviction means removing entries from the cache. This can happen because memory is full, an entry has expired, or the cache decides an entry is no longer worth keeping.

An eviction policy answers one question: when the cache needs space, which entry should go?

There is no perfect answer for every system. A policy that works well for product pages may work poorly for analytics queries, search results, or one-time background jobs. The choice directly affects cache hit rate, latency, and database load.

This chapter covers why eviction policies matter, the most common policies like LRU, LFU, FIFO, random replacement, MRU, and TTL, and how to choose and monitor them in a real system.

Eviction affects more than memory usage. It affects latency, database load, and user experience.

If the cache removes useful entries too quickly, the hit rate drops and more requests go to the database. If the cache keeps the wrong entries, memory fills with data nobody asks for anymore.

Good eviction keeps the right data in memory.

The working set is the data your application is likely to request again soon. Every eviction policy is basically a different guess about what belongs in that working set.

Least Recently Used, or LRU, removes the entry that has gone the longest without being read or written.

The idea is simple: if something was used recently, it is more likely to be used again soon.

How It Works

On every read or write, the cache marks that entry as recently used. When memory is full, the cache removes the entry that has waited the longest since its last use.

Example with capacity 3:

Trade-offs

LRU is easy to understand and works well for many web and API workloads. It keeps recently active data in memory and adapts quickly when traffic changes.

The trade-off is that the cache must track when each entry was last used. LRU can also perform poorly during large one-time scans. A scan can make many cold entries look recent and push out data users actually need.

Best Fit

LRU is a strong default for general-purpose application caches where recently used data is likely to be used again soon.

It can struggle with scan-heavy workloads. For example, a one-time job that reads millions of keys can push useful hot data out of the cache.

Least Frequently Used, or LFU, removes the entry with the lowest access count.

The idea is that data read many times is usually more valuable than data read once.

How It Works

The cache tracks roughly how often each entry is used. When memory is full, it removes an entry with the lowest count. If two entries have the same count, the cache usually removes the one used less recently.

Trade-offs

LFU protects keys that stay popular over time. It works well when long-term popularity matters more than recent activity.

The trade-off is that the cache must track usage counts. Also, an item that was popular yesterday may stay in the cache too long after users stop asking for it. Real LFU implementations usually reduce old counts over time, so old traffic does not matter forever.

Best Fit

LFU works well when a small set of keys stays popular for a long time, such as product catalogs, profile details, configuration, and common lookup tables.

It works poorly when popularity changes quickly unless old counts fade over time.

First In, First Out, or FIFO, removes the entry that was inserted first.

It does not care whether the entry was accessed recently or often.

How It Works

Entries are kept in insertion order. When the cache is full, the oldest inserted entry is removed.

Trade-offs

FIFO is very simple and predictable. It does not need much extra tracking.

The downside is that it ignores how the cache is actually used. It can remove a popular entry just because that entry was inserted early. For most application caches, FIFO usually has a lower hit rate than LRU or LFU.

Best Fit

FIFO is useful when simplicity matters more than hit rate, or when cached entries have similar value and similar access patterns.

It is rarely the best policy for high-traffic application data.

Random replacement removes a randomly selected entry when space is needed.

It does not track recency, frequency, or insertion order.

How It Works

When the cache is full, the cache picks one entry at random and removes it.

Trade-offs

Random replacement needs almost no extra tracking, so it is simple and fast. It can work surprisingly well when access patterns are messy or hard to predict.

The downside is obvious: it can remove hot data by bad luck. It is also harder to reason about because the same workload may behave differently from run to run.

Best Fit

Random replacement can be acceptable when the cache must stay very lightweight or access patterns are unpredictable.

It is also useful as a baseline. If LRU or LFU is only slightly better than random, the extra complexity may not be worth it.

Most Recently Used, or MRU, removes the entry that was used most recently.

This sounds strange because most systems want the opposite. MRU is useful only for specific workloads where the most recent item is unlikely to be used again.

How It Works

The cache tracks which entry was used most recently. When space is needed, that entry is removed.

Trade-offs

MRU can help with some scan or repeated-cycle access patterns. It keeps older entries that may still be useful.

The downside is that it is a poor default for most application caches. In normal web workloads, recently used data is often exactly the data you want to keep.

Best Fit

MRU can fit workloads where recently read data is unlikely to be reused, such as certain sequential scans or repeated cycles.

For most web services, LRU is a better starting point.

Time-To-Live, or TTL, expires entries after a set amount of time.

TTL is often discussed with eviction policies, but it solves a slightly different problem. LRU and LFU decide what to remove when memory is tight. TTL decides how long data is allowed to stay in the cache.

How It Works

Each entry gets an expiration time.

Expired entries may be removed by a background cleanup process, or they may be removed the next time someone tries to read them.

Trade-offs

TTL limits how long stale data can live in the cache. It also cleans up entries that may never be read again, and it is simple to configure.

The trade-off is that TTL can remove hot data even when memory is still available. TTL also does not choose the best entry to remove when memory is full. And if many keys have the same TTL, they can expire together and cause a sudden spike in cache misses.

Best Fit

Use TTL when cached data has a freshness requirement, meaning old data should not be served forever.

For production systems, add jitter so many entries do not expire at the same time:

The random offset turns one big expiration wave into a steadier stream of refreshes that the database can handle.

Start with the workload, not the policy name. Ask what kind of data should stay in memory, and what kind of data is safe to remove.

Modern caches often use practical versions of these ideas rather than pure textbook policies. Redis supports policies such as allkeys-lru, volatile-lru, allkeys-lfu, volatile-ttl, and several random variants. Many local caches use LRU-like policies that protect hot data from one-time scans. Some systems combine TTL with recency or frequency tracking.

Do not memorize policy names as if one policy wins everywhere. Measure hit rate and eviction behavior under the actual workload.

Eviction problems often come from measuring the wrong thing or caching data that should not be cached in the first place.

Evicting by Count Instead of Size

One large value can use as much memory as thousands of small values.

If the cache limits only the number of items, a few large entries can crowd out many useful small entries. Track memory usage and item sizes, not only key count.

Caching One-Time Reads

Caching data that will never be read again wastes cache space. Common examples include large report exports, one-off admin queries, full table scans, backfills, and migrations.

Bypass the cache for known one-time access patterns.

Ignoring Negative Results

If many requests ask for data that does not exist, the database can still be overloaded.

Short-lived negative caching can help:

Keep the TTL short so newly created data does not stay hidden behind an old "not found" cache entry.

Treating TTL as a Memory Policy

TTL helps with freshness. It does not guarantee that the most useful entries stay in memory.

Use TTL with a memory eviction policy when both freshness and memory pressure matter.

Not Reserving Headroom

Running a cache too close to its memory limit causes constant evictions.

Constant eviction means the cache is churning: entries are inserted and removed before they can help much.

You cannot tune eviction by looking only at cache hit rate. Hit rate matters, but it does not tell the whole story.

Track several metrics together. Start with hit rate by route or key pattern, so you know whether important paths benefit from the cache. Then watch eviction rate, which tells you whether memory pressure is active.

Also watch memory usage, item sizes, and miss rate after eviction. These show whether the cache is close to capacity, whether large values are crowding out smaller ones, and whether evictions are hurting users.

Finally, watch database QPS, or queries per second, after evictions and the TTL expiry rate. These tell you whether the database is being exposed and whether expirations are causing miss spikes.

Useful alerts fire when the eviction rate spikes suddenly, hit rate drops on critical key patterns, memory usage stays near the limit, database QPS rises after eviction increases, or many keys expire at the same time.

Cache eviction decides what leaves the cache when space is needed. LRU removes the least recently used entry and is a good default. LFU removes the least frequently used entry and protects data that stays popular. FIFO removes the oldest inserted entry, which is simple but often less effective.

Random eviction is cheap and sometimes acceptable. MRU removes the most recently used entry and helps only for specific patterns. TTL expires entries by age, which helps with freshness but is not a complete memory policy on its own.

Choose based on workload, then verify with metrics: hit rate, eviction rate, memory usage, item sizes, and database load. The best policy keeps the working set in memory without letting stale or low-value data crowd out the entries users need.