In the previous chapter, we explored document databases and their flexible JSON-based data model.

While document databases offer rich query capabilities over nested structures, sometimes you do not need any of that. Sometimes you just need to store a value and retrieve it later by a key. No queries. No indexes. No schema. Just pure, fast storage and retrieval.

This is the domain of key-value stores. They are the simplest database type: give it a key and a value, get the value back when you provide the key.

By doing less, key-value stores can be incredibly fast. We are talking sub-millisecond latency for reads and writes. Redis, the most popular key-value store, can handle over a million operations per second on a single server.

Think of a key-value store like a hash map that persists to disk. You have used hash maps in your code: constant-time lookups, simple API, no complex queries. Key-value stores bring that model to the database layer, with the addition of persistence, replication, and distributed scaling.

The Key-Value Model

A key-value store is, at its core, a giant hash map. You store data as key-value pairs:

Operations

The API is minimal:

Some stores add variations:

What Makes Key-Value Stores Fast

Several design choices contribute to sub-millisecond performance:

1. Simple data access pattern. There is no query parsing, query planning, or index traversal. The key hashes directly to a memory location.

2. In-memory storage. Many key-value stores (Redis, Memcached) keep all data in RAM. Memory access is orders of magnitude faster than disk.

3. No schema overhead. Values are opaque blobs. The database does not parse, validate, or index them.

4. Minimal coordination. Single-key operations do not need locks or transaction coordination in most cases.

Key Design Best Practices

Keys are your only access path.

Use meaningful prefixes

Keep keys reasonably short

Keys consume memory. A key like user:1001 uses 10 bytes. A key like application:production:user:profile:identifier:1001 uses 51 bytes. At millions of keys, this adds up.

Use consistent separators

Colons (:) are conventional, but be consistent. Some tools use dots or slashes.

Avoid special characters

Stick to alphanumeric characters and simple separators for compatibility.

Common Use Cases

Key-value stores solve specific problems exceptionally well. Understanding these patterns helps you recognize when to reach for them.

Caching

The most common use case. Place a key-value store in front of your primary database to cache frequently accessed data:

Cache-aside pattern (most common):

Benefits:

Session Storage

User sessions need fast access on every request. Storing them in a key-value store is ideal:

Why key-value for sessions:

Rate Limiting

Limit API requests per user or IP address:

Why key-value for rate limiting:

Real-Time Leaderboards

Sorted sets in Redis make leaderboards trivial:

Why key-value (Redis) for leaderboards:

Shopping Carts

Shopping carts are temporary, per-user, and accessed frequently:

Why key-value for carts:

Pub/Sub Messaging

Redis supports publish/subscribe messaging:

Why key-value (Redis) for pub/sub:

Redis Deep Dive

Redis is the most popular key-value store, powering critical systems at Twitter, GitHub, Snapchat, and countless others. While it started as a simple key-value cache, it has evolved into a "data structure server" with rich capabilities.

Data Types

Redis goes beyond simple strings. Each data type has specialized commands:

String Operations

Beyond GET/SET, strings support:

List Operations

Lists are doubly-linked lists, efficient for push/pop at either end:

Use case: Job queue

Hash Operations

Hashes store objects without serialization overhead:

Why hashes vs. JSON strings:

Sorted Set Operations

Sorted sets maintain elements ordered by score:

Expiration (TTL)

Keys can have automatic expiration:

TTL is essential for:

Transactions

Redis transactions group commands to execute atomically:

Commands are queued during MULTI and executed atomically on EXEC. However, Redis transactions are not rollback-safe. If a command fails, others still execute.

For true atomicity, use Lua scripts:

In-Memory vs Persistent

Key-value stores fall into two categories:

In-Memory (Redis, Memcached)

Data lives primarily in RAM with optional persistence to disk.

Redis persistence options:

Persistent (DynamoDB, etcd)

Data is durably stored to disk with replication for fault tolerance.

Comparison

Distributed Key-Value Stores

For scale beyond a single server, key-value stores can be distributed across multiple nodes.

Redis Cluster

Redis Cluster automatically partitions data across multiple nodes:

How it works:

Limitations:

Amazon DynamoDB

DynamoDB is a fully managed key-value and document database:

Key features:

Data model:

etcd

etcd is a distributed key-value store designed for configuration and coordination:

Use cases:

Characteristics:

Consistency Models

Distributed key-value stores offer different consistency guarantees:

Key-Value Store Patterns

Cache-Aside (Lazy Loading)

The application manages the cache explicitly:

Pros: Simple, cache only what is accessed, tolerant of cache failures. Cons: First request is slow (cache miss), potential stale data.

Write-Through

Write to cache and database together:

Pros: Cache is always consistent with DB. Cons: Write latency includes both DB and cache, cache may store rarely-read data.

Write-Behind (Write-Back)

Write to cache immediately, persist to database asynchronously:

Pros: Very fast writes, can batch database operations. Cons: Risk of data loss if cache fails before persist, complexity.

Distributed Locking

Use Redis for distributed locks:

For production use, consider Redlock algorithm or Redis RedLock for multi-node scenarios.

When to Choose Key-Value Stores

Key-value stores are the right choice when:

When to Consider Alternatives

Key-value stores may not fit when:

Summary

Key-value stores offer the simplest data model and the fastest performance:

Key use cases:

The next chapter explores wide-column stores, which extend the key-value model with column families for handling massive scale and high write throughput.