Reading from a database or calling another service takes time. Under heavy traffic, that cost adds up quickly.

A cache is a fast storage layer that keeps copies of frequently used data. When the application needs that data again, it can read from the cache instead of going all the way back to a slower system.

This chapter breaks down caching: how it works, why systems use it, how to measure it, and when to add a cache in a system.

At its core, caching is simply the idea of storing data in a faster location so you can reuse it later.

Suppose your application needs to fetch a product from a database. The first time a user requests that product, the application queries the database, gets the result, and returns it.

But if thousands of users request the exact same product again and again, querying the database every single time is wasteful. Instead, we can store that result in a cache.

Now, when the next request comes in, the application first checks the cache. Two things can happen:

The flowchart below shows both paths for a single request:

A miss costs a little more than a request with no cache at all, because the application checks the cache first. The payoff comes from the requests after it: every one of them for the same product is now a hit.

The main reason we use caching is performance. It gives us three major benefits.

First, lower latency. Caches usually store data in much faster storage, often memory, so reading from the cache can be significantly faster than querying a database or calling another service.

Second, reduced load. Because fewer requests reach the database or downstream services, those systems have more capacity to handle other work.

And third, higher throughput. If most requests are served cheaply from the cache, the overall system can handle much more traffic with the same underlying infrastructure.

So caching is not just about making individual requests faster. It also helps protect expensive components like databases and makes the entire system easier to scale.

One of the most important metrics for any cache is the cache hit ratio. It tells us what percentage of requests are successfully served from the cache.

The formula is simple:

Suppose your application receives one million requests and 900,000 of them are served from the cache. That gives you a 90 percent hit ratio.

In general, the higher the hit ratio, the more work the cache is saving. But the number has to be understood in context.

A 95 percent hit ratio may sound excellent. But if your system handles 10 million requests per second, the remaining five percent still means 500,000 requests per second are reaching the backend:

So even a relatively small miss rate can create significant load at scale. When you size a database behind a cache, size it for the misses, not for the total traffic.

Caching can happen at many different layers of a system. So a request might pass through several caching layers before it reaches the database:

Each layer has a different job.

Browser Cache

Your browser can cache images, JavaScript, CSS, and even API responses. It is the closest cache to the user. If the browser already has a valid copy, it may not need to make a network request at all.

Behavior is controlled through HTTP headers like Cache-Control and ETag:

CDN Cache

A CDN can cache content close to users around the world. Users download content from a nearby edge location instead of from your main servers. CDNs work best for static content, media files, public pages, and other content that many users can share.

Application Cache

Your application server can keep frequently accessed data directly in memory. It is very fast because the data is already inside the running process.

The trade-off is that each application instance has its own local cache. That can duplicate memory and create different cached values on different instances.

Distributed Cache

Multiple servers can share a distributed cache such as Redis or Memcached. Lookups are slower than local memory because they cross the network, but every server sees the same cached data, and the cache can hold much more of it.

Database Cache

Even databases maintain internal caches for frequently accessed pages and query data. Repeated reads of the same pages are served from memory on the database server instead of from disk. This layer is usually invisible to application code; the database manages it automatically.

Not every piece of data should be cached. Caching works best when the same data is requested repeatedly and is relatively expensive to fetch or compute, like:

The more frequently a value is reused, the more useful caching becomes.

On the other hand, some data is a poor fit:

If a value changes constantly, the cached copy may become stale almost immediately. If the data requires very strong freshness guarantees, serving an outdated value from cache may be unacceptable. You should also think about size. Caching a huge object that gets requested once every few hours may waste memory without providing much benefit.

So before adding a cache, ask:

Every cached value needs a way to identify it. That identifier is called a cache key. At its core, a cache is usually a key-value store: the application gives the cache a key, and the cache returns the value stored under it.

For example, if we want to cache product 123, the key might be product:123. For a user, user:456. For a paginated search result, the key might include the search term and page number, like search:shoes:page:2.

Good cache key design is important because the key determines whether two requests should reuse the same cached value.

Keys should usually include every parameter that changes the result. A request for shoes sorted by price produces a different result than one sorted by rating, so both parameters belong in the key:

It is also common to use namespaces such as user, product, or session. And sometimes a version is included in the key, which makes it easier to invalidate entire groups of entries when a format changes. When the product format changes, the application starts reading v2: keys, and every v1: entry stops being used at once.

A cached value usually should not live forever. That is why caches often use a TTL, or Time To Live. The TTL defines how long a cache entry remains valid before it expires.

Suppose we cache product 123 for five minutes. During those five minutes, requests can be served directly from the cache. Once the TTL expires, the next request will usually fetch fresh data from the database and repopulate the cache.

Choosing the right TTL is a trade-off:

So there is no universal best TTL. It depends on how quickly the data changes and how much staleness the feature can tolerate:

Caching is powerful, but you should not automatically add it to every system. A cache introduces extra complexity. Now you have to think about expiration, invalidation, stale data, failures, and consistency.

So before adding one, first ask whether there is actually a performance problem to solve. If your database queries are already fast and traffic is low, a cache may provide very little benefit.

Caching becomes valuable when the same data is requested repeatedly, when the underlying operation is expensive, or when the database or downstream service is becoming a bottleneck. You should also ask whether the data can tolerate some staleness.

A good rule is to look for three things:

If those conditions exist, caching can be extremely effective.

A cache stores data in a faster location so the system can reuse it. When the data is in the cache, the request is a hit and never touches the database. When it is not, the request is a miss: the application fetches the data, stores a copy, and returns it.

Caching lowers latency, reduces load on databases and downstream services, and raises throughput. The hit ratio measures how much work the cache saves, but it has to be read in context: at 10 million requests per second, a 95 percent hit ratio still sends 500,000 requests per second to the backend.

Caches exist at many layers: the browser, the CDN, the application's memory, a shared distributed cache, and inside the database itself. Data that is requested often and expensive to get is a good fit. Data that changes constantly, must always be fresh, or is huge and rarely read is not.

Cache keys decide which requests share a value, so they should include every parameter that changes the result. TTLs decide how long a value lives, trading freshness against hit ratio. And a cache adds real complexity, so add one only when there is a performance problem it can solve.

The rest of this section goes deeper into the most important caching concepts and patterns you will encounter in real systems, starting with the most common one, the cache-aside pattern, where the application explicitly manages what goes in and out of the cache.