A cache keeps frequently used data in fast storage, usually memory, so the application does not have to recompute it or fetch it from a slower database on every request.

A single cache node is often enough for small systems. It is simple, fast, and easy to operate.

As the system grows, one cache node can become a limit. It may not have enough memory for all the hot data. It may not handle the request volume. It can also become a single point of failure.

Distributed caching solves this by spreading cached data across multiple cache nodes.

In this chapter, we will cover what distributed caching means, why systems use it, how keys are placed across cache nodes, the difference between shared and local cache designs, common problems, practical production habits, and popular caching technologies.

A distributed cache stores cached data across multiple machines instead of keeping everything on one machine.

From the application's point of view, it still looks like a key-value cache:

Behind the scenes, the cache client or cache service decides which node should store product:123.

A distributed cache is usually split across nodes, not copied fully onto every node.

If you have 100 GB of cached data and 5 cache nodes, each node might hold about 20 GB. That is how the cache grows beyond the limits of one machine.

Some systems also keep extra copies for availability. That can make failures easier to handle, but it uses more memory and adds complexity.

Distributed caching is useful when one cache node is no longer enough.

More Memory

Each cache node contributes memory to the cluster. Adding nodes lets the cache hold a larger working set.

This matters when the application repeatedly reads millions of products, profiles, permissions, search results, feed items, or computed responses.

More Throughput

Requests are spread across nodes, so no single cache process has to serve every read and write.

This helps when cache traffic is high enough that one node runs out of CPU, network bandwidth, or available connections.

Better Availability

With multiple nodes, the failure of one cache node does not have to take down the whole cache layer.

There is still a cost. Keys on the failed node may be unavailable, cold, or served from another copy depending on the cache design. The application must still handle misses and fall back to the database.

Lower Database Load

The main reason to cache is still the same: protect slower dependencies.

A well-designed distributed cache can absorb a large share of read traffic and keep the database focused on writes, cache misses, and queries that truly need fresh data.

Most distributed caches follow the same basic flow:

Key Distribution

The cache has to answer one question for every key:

Which node should store this key?

There are two common approaches:

Modulo hashing is easy to understand, but it behaves badly when the cluster size changes. If you move from 5 nodes to 6 nodes, many keys move to different nodes. That creates lots of cache misses at once.

Consistent hashing reduces that disruption. When a node is added or removed, only a smaller portion of keys move. This is why consistent hashing is common in distributed caches and storage systems.

Sharding

Sharding means splitting the cache keys across multiple nodes.

For example:

Sharding is what lets a distributed cache grow beyond the memory and request capacity of one node.

Replication

Replication means keeping a copy of a cache entry on more than one node.

Replication can improve availability. If the main node for a key fails, another node may still have a copy.

But replication is not free:

For many cache-aside systems, losing cached data is acceptable because the database still has the real data. In those systems, replication is useful but not always required.

Eviction

Caches have limited memory. When memory fills up, the cache must remove something. Common policies include LRU, LFU, TTL-based expiration, and size-based limits.

Eviction is normal. Applications should treat the cache as a performance optimization, not as the only copy of important data.

One design decision is where the cache runs.

The two common options are dedicated cache servers and co-located caches.

Dedicated Cache Servers

Dedicated cache servers run separately from application servers.

This is the most common design for shared caches such as Redis, Valkey, or Memcached.

Co-Located Cache

A co-located cache runs on the same machine or in the same process as the application.

This is often called an in-process cache or local cache.

It is fast because it avoids a network call. It is useful for small, frequently read data such as feature flags, routing tables, permissions, templates, or reference data.

Multi-Level Caching

Large systems often stack two cache layers in front of the database. An L1 cache is a small local cache inside each application instance. An L2 cache is the shared distributed cache that every instance can reach. The database remains the real data source behind both.

This design can work well, but cache invalidation becomes harder. If data changes, you may need to delete or refresh both the shared cache entry and every local copy.

Distributed caching improves scale, but it also introduces new ways things can go wrong.

Cache Invalidation

Invalidation means deleting or refreshing cached data when the real data changes. It is often the hardest part of caching.

If the database changes but the cache still returns the old value, users see stale data. In distributed systems, this gets harder because the value may exist in several places: local caches, shared cache nodes, replicated copies, and client-side caches.

Common approaches include TTL-based expiration, deleting the cache entry when data is written, versioned cache keys, sending cache-delete events through a message queue, and using short TTLs for sensitive data. There is no universal answer. The right choice depends on how old the cached data is allowed to be.

Hot Keys

Even with many cache nodes, traffic may not be evenly distributed.

A single celebrity profile, flash-sale product, leaderboard, or global configuration key can receive a large share of traffic. The node holding that key becomes overloaded while the rest of the cluster is underused.

Common fixes include copying very hot keys to more than one node, adding short-lived local caching, splitting large values into smaller keys, rate limiting expensive rebuilds, and making only one request rebuild a missing value while the others wait.

Rebalancing

When nodes are added or removed, some keys move.

Moving keys can create cache misses, uneven load, and extra traffic to the database. Consistent hashing reduces the damage, but it does not eliminate it.

During rebalancing, watch cache hit rate, database queries per second, latency, and error rates.

Network Failures

A distributed cache depends on the network. The application must handle timeouts, too many cache connections being used, partial cache outages, slow cache nodes, and failure handling in replicated systems.

Cache calls should have tight timeouts. A slow cache should not be allowed to make the whole application slower than going directly to the database.

Memory Pressure

When memory fills up, the cache removes entries. If this happens too often, hit rate drops and database load rises.

Large values can make this worse. A few oversized entries can push out many useful small entries.

Track memory usage, eviction rate, item sizes, and hit rate together. Looking at only one metric can hide the real problem.

Consistency Expectations

A cache is usually not the real data source.

Most systems accept some stale data in exchange for speed. Problems happen when the application expects database-like freshness from a cache that was designed for performance.

Be explicit about which data can be stale and for how long.

Running a distributed cache well comes down to a few habits that hold up under load and failure.

Cache Data That Is Worth Caching

A good fit for caching is data that is read often, expensive to fetch or compute, small enough to store efficiently, safe to serve slightly stale, and shared by many requests. Poor fits are rarely read, cheap to fetch, very large, highly sensitive to stale data, or unique to a single request.

Use Clear Cache Keys

Good keys are predictable, grouped by purpose, and easy to version.

Versioned keys are useful when the stored value changes shape. Instead of trying to delete every old key, the application starts writing and reading a new key prefix.

Set TTLs Intentionally

TTL is a product and operations decision, not only a memory setting.

Use shorter TTLs for data that changes often or must not be stale for long. Use longer TTLs for stable reference data.

Add jitter to avoid many keys expiring at the same time:

Protect the Database on Misses

Cache misses are normal. Miss storms are dangerous. Use techniques such as letting one request rebuild a value while others wait, warming critical keys ahead of time, rate limiting expensive rebuilds, stopping calls to overloaded dependencies temporarily, and serving stale data briefly when availability matters more than perfect freshness.

Design for Cache Failure

The application should continue to work when the cache is slow, empty, or partly unavailable. That means tight cache timeouts, connection limits, fallback to the database, simpler behavior for non-critical features, and alerts on hit rate, latency, errors, memory usage, and evictions.

Keep Values Small

Large cache values increase network cost, encoding and decoding cost, memory pressure, and eviction damage.

Cache the shape the application needs, but avoid turning the cache into a dumping ground for huge objects.

Monitor by Route and Key Pattern

Global hit rate can be misleading.

A 95% hit rate may still hide a bad miss rate on checkout, login, or search. Track hit rate, latency, and error rate by route, tenant, key pattern, and cache node.

A handful of systems show up repeatedly when teams build distributed caches. Each one has different trade-offs around data structures, persistence, licensing, and how much work it takes to run well.

Memcached

Memcached is a high-performance distributed memory object cache. It is simple, fast, and commonly used for small chunks of arbitrary data such as database query results, API responses, and rendered page fragments.

Memcached is a good fit when you want a simple temporary cache and do not need disk persistence, replication features, or rich data structures.

Redis

Redis is an in-memory data store that supports strings, hashes, lists, sets, sorted sets, streams, geospatial indexes, and other data structures.

Redis is often used as a cache, but it is also used for rate limiting, queues, leaderboards, sessions, and lightweight coordination. Because Redis can do much more than a simple cache, teams should be clear about whether the data is disposable cache data or important workflow data.

Redis licensing has shifted twice in recent years. Redis 7.4 moved from the BSD 3-clause license to a dual SSPL and RSALv2 model in early 2024.

Redis 8 added AGPLv3 as an additional option in 2025, so current Redis releases are once again available under an OSI-approved open-source license. Teams with open-source or managed-service requirements should still review the license terms before choosing a deployment model.

Valkey

Valkey is an open-source, Redis-compatible in-memory data store created after the Redis licensing change.

It is relevant when teams want Redis-compatible behavior under an open-source project model.

Managed Cache Services

Cloud providers offer managed cache services that handle setup, patching, monitoring integrations, backups where supported, and failover options.

For example, Amazon ElastiCache supports Valkey, Memcached, and Redis OSS engines.

Managed services reduce the work of running the cache, but they do not remove the design work. You still need good keys, TTLs, capacity planning, failure handling, and an invalidation strategy.

Distributed caching stores cached data across multiple machines so the cache layer can scale beyond a single node.

The main building blocks are key distribution, sharding, optional replication, eviction, and client routing. The main design choices are whether to use dedicated cache servers, local caches, or a multi-level L1/L2 design.

Distributed caches reduce latency and database load, but they introduce trade-offs around invalidation, hot keys, rebalancing, network failures, memory pressure, and stale data.

Treat the cache as a fast, partial, temporary copy of data, designed so it can be empty, stale, slow, or missing without taking the whole system down.