System design interviews test your knowledge of how large-scale systems work. A solid grasp of the core concepts helps you design a system clearly and explain why each choice makes sense.

The good news is that most system design problems reuse the same core ideas. Whether you are designing a URL shortener, a chat app, a news feed, or a payment system, the building blocks are largely the same.

In this chapter, we will cover the essential system design concepts you need to know, organized by category. For each concept, we will look at what it means, why it matters, and when it is useful in a real design.

Most system design discussions eventually reach the same question: What happens when traffic grows?

A design that works for a thousand users may fail badly at a million. The database may slow down, server may run out of CPU, queues may start backing up, and latency may keep rising. Scalability is about understanding where those limits appear and how to move them.

A simple design is usually the right starting point. You do not need to design for global scale on day one. But you do need to know what you would change as the system grows.

There are two common ways to scale a system:

a. Vertical Scaling

Vertical scaling, also called scaling up, means adding more power to the existing machine.

For example, you might move from a small server to one with more CPU, more RAM, faster disks, or better network capacity.

This is often the easiest first step because the application does not need major changes. You keep the same basic architecture and run it on a stronger machine.

But vertical scaling has limits. A single machine can only get so big, and high-end machines become expensive quickly. It also leaves you with a major risk: if that one machine fails, the system can go down.

b. Horizontal Scaling

Horizontal scaling, also called scaling out, means adding more machines and spreading the load across them.

Instead of one large server handling everything, you run multiple servers behind a load balancer. Each server handles part of the traffic.

This is how most large systems scale over time. It gives you more room to grow and improves availability because one failed server does not have to bring down the whole system.

The trade-off is complexity. Your application usually needs to be stateless, requests need to be routed correctly, data may need to be shared or partitioned, and failures become harder to reason about.

In practice, many systems start with vertical scaling because it is simple. You upgrade the server, increase database capacity, or add more memory.

But once one machine is no longer enough, you move toward horizontal scaling. That usually means adding load balancers, making services stateless, using caches, adding replicas, and eventually splitting data across machines.

The key idea is simple: scale vertically when simplicity matters, and scale horizontally when one machine is no longer enough.

Load Balancing

Once you run multiple servers, you need a way to spread traffic across them.

A load balancer sits in front of your servers. It receives incoming requests from clients and decides which backend server should handle each request.

Without a load balancer, clients would need to know about every server. That is hard to manage and breaks easily when servers are added, removed, or replaced.

A load balancer gives you one stable entry point and hides the backend server changes from clients.

Common Load balancing algorithms

Round Robin is the simplest approach and is often good enough when all servers are similar and requests take roughly the same amount of time.

But if some requests are slow and others are fast, Round Robin can create uneven load. One server may end up stuck with several long-running requests while another server stays mostly free. In that case, Least Connections usually works better because it looks at how busy each server is right now.

Here are few more things to know about load balancers:

Health Checks

Load balancers also perform health checks. They regularly call each server to check whether it is alive and able to serve traffic. If a server stops responding or starts failing health checks, the load balancer removes it from rotation.

This gives you basic automatic failover. A failed server does not have to take down the whole system.

Sticky Sessions

Sometimes the same user needs to keep going to the same backend server. This is called a sticky session.

You usually need sticky sessions when session data is stored locally on the server. For example, if Server 1 stores a user's login session in memory, sending the next request to Server 2 may break the flow.

One way to support this is IP Hash, where requests from the same client IP usually go to the same server. However, a better long-term design is to keep servers stateless. Store session data in a shared place, such as Redis or a database, so any server can handle any request.

Layer 4 vs Layer 7 Load Balancing

Load balancers usually operate at either Layer 4 or Layer 7.

Layer 4 load balancing works at the TCP/UDP level. It routes traffic based on IP address and port. It is fast because it does not inspect the HTTP request.

Layer 7 load balancing works at the HTTP level. It can route traffic based on request paths, headers, cookies, or hostnames.

For example:

Layer 7 is more flexible, but it does more work per request. Layer 4 is simpler and faster, but less aware of application details.

SSL Termination

Many load balancers also handle SSL termination.

This means the client connects to the load balancer over HTTPS, and the load balancer decrypts the request. After that, it forwards the request to backend servers.

This keeps certificate management in one place and reduces encryption work on each backend server.

In production, backend traffic is often still encrypted, especially across networks or cloud boundaries. But the key idea remains the same: the load balancer often becomes the place where HTTPS is handled before traffic reaches the application servers.

Auto Scaling

Traffic is rarely steady.

An application may need only a few servers late at night, but many more during peak hours. Auto scaling adjusts capacity automatically. It adds servers when load increases and removes them when load drops.

Auto scaling usually works by watching system metrics and applying rules.

Common triggers include:

For example, you may decide to add two more servers when average CPU stays above 70% for five minutes. You may remove servers when CPU stays low for a longer period.

The hard part is not turning auto scaling on. The hard part is choosing the right signals and thresholds.

If you scale out too aggressively, you waste money. If you scale out too slowly, users see high latency or failed requests during traffic spikes.

Most production systems set both a minimum and maximum number of instances.

The minimum keeps enough capacity available even during quiet periods. It also protects you from scaling down too far.

The maximum prevents runaway cost. Without a limit, a bug, traffic spike, retry storm, or attack can keep creating more servers than you intended.

Auto scaling is useful, but it is not instant. New servers need time to start, warm up, load configuration, connect to dependencies, and begin receiving traffic. For predictable traffic spikes, teams often scale ahead of time instead of waiting for the system to react.

Database choice should come from the system requirements, not personal preference.

In a system design interview, do not start with, “I will use PostgreSQL” or “I will use Cassandra.” Start with the data:

Once those answers are clear, the database choice becomes much easier to justify.

SQL vs NoSQL

This is one of the most common database decisions in system design interviews.

SQL databases, such as MySQL and PostgreSQL, are a strong fit when the data is structured and relationships matter.

They give you:

They work well for systems like payments, orders, inventory, bookings, user accounts, and internal business tools.

NoSQL databases, such as MongoDB, Cassandra, DynamoDB, and Redis, are useful when the access pattern is simpler or the scale requirements are different.

They can give you:

They work well for use cases like session storage, user timelines, event logs, product catalogs, analytics data, and high-volume key-value lookups.

A good interview rule of thumb is this: Start with a relational database unless you have a clear reason not to.

Relational databases are boring in the best way. They handle transactions, constraints, indexes, and queries well. For many systems, that is exactly what you want.

Move away from SQL when the requirements justify it. Common reasons include massive write volume, simple key-value access at very high scale, document-shaped data, graph traversal, or multi-region availability where eventual consistency is acceptable.

Database Indexing

Without an index, the database may have to scan every row to find the data you asked for.

For example, if you search for a user by email and there is no index on the email column, the database may check row after row until it finds a match.

Most relational database indexes are backed by a B-tree or a similar structure. You can think of it like a phone book. Instead of reading every name from the beginning, you jump to the right section and narrow down from there.

Common Types of Indexes

When to add an index

Indexes are usually useful on columns that appear often in:

For example, indexing email usually helps because emails are unique. Indexing a boolean column like is_active is often less useful because there are only two possible values.

The practical rule is simple: add indexes based on real query patterns, not guesswork. An index should exist because a query needs it.

Database Replication

A single database can become a single point of failure. If it goes down, the application may stop working.

Replication reduces that risk by keeping copies of the same data on multiple database servers. If one server fails, another copy can continue serving traffic.

A common setup looks like this:

In this model, the primary handles writes. The replicas receive changes from the primary and can serve read traffic.

This improves availability and helps scale reads, but it introduces an important question: When should the primary consider a write successful?

Types of Replication

Many systems use asynchronous replication because it keeps write latency low. The trade-off is that replicas may temporarily be behind the primary.

This delay is called replication lag.

Replication lag can create confusing user experiences. For example:

This is why read-your-writes consistency matters. After a user successfully writes data, their next read should show that change.

A common fix is to route a user's reads to the primary for a short time after they write, or to track replica lag and only read from replicas that are caught up.

Replication mainly helps with three things. It improves high availability, since a replica can be promoted if the primary fails. It improves read scalability, since read traffic can be spread across replicas. And it supports geographic distribution, since replicas can be placed closer to users in different regions.

Replication is one of the first tools to consider when a database needs better availability or more read capacity. But it has costs. You now have to think about lag, failover, consistency, and what happens when replicas disagree or fall behind.

Database Sharding

Replication helps you scale reads, but it does not remove the write bottleneck. In a primary-replica setup, every write still goes to the primary.

So what happens when one database can no longer handle the write traffic? Or when the data no longer fits comfortably on one machine?

That is where sharding comes in.

Sharding means splitting data across multiple databases. Each database holds only part of the data.

The application, or a shard router, decides which shard should handle each request.

The hard part is choosing the sharding key. This is the field used to decide where each record lives. For a user-based system, it might be user_id. For an order system, it might be customer_id or merchant_id.

A good sharding key should spread data evenly and keep related data together.

Common Sharding Strategies

Range-based sharding is easy to understand, but it can create hot spots. For example, if most new users fall into one range, that shard gets more traffic than the others.

Hash-based sharding is often a better default because it spreads data more evenly. The downside is that related ranges are no longer stored together, so range queries may need to touch many shards.

A simple hash-based approach is:

This works, but it has one big problem. When you add or remove shards, the number of shards changes, and many keys move to different places. That can force a large data migration.

Consistent Hashing

Consistent hashing reduces this problem.

Instead of mapping keys directly using hash(key) % N, consistent hashing places both shards and keys on a hash ring. Each key belongs to the next shard on the ring.

When you add or remove a shard, only part of the key space moves. The rest stays where it is. That makes scaling less painful, especially in systems where shards are added or removed over time.

Costs of Sharding

Sharding is powerful, but it adds real complexity.

Because of these costs, sharding is usually not the first step.

A practical order is:

Database reads are much slower than memory lookups. Even with good indexes, a database query may take milliseconds. A cache lookup is usually much faster.

At small scale, that difference may not matter much. At thousands of requests per second, it does.

Caching means storing frequently accessed data in memory so the application does not have to query the database every time.

For example, instead of reading a user's profile, product details, feed items, or configuration from the database on every request, you can keep a copy in cache and serve it quickly.

Beyond "Should we cache this?", the harder question is: How do we keep the cache reasonably fresh without making the system too complex?

Cache Strategies

The biggest challenge with caching is keeping the cache and database in sync. Different caching patterns make different trade-offs between freshness, latency, and reliability.

Cache-Aside

Cache-aside, also called lazy loading, is the most common caching pattern. In this approach, the application manages the cache directly.

The application first checks the cache. If the data is present, it returns it. If not, it reads from the database, stores the result in cache, and returns the data.

This is a good default for most interview problems because it is simple and works well for read-heavy systems.

Other Caching Patterns

Cache Invalidation

Cache invalidation is hard because cached data can become stale.

Suppose a user updates their profile in the database. If the old profile is still sitting in cache, the application may keep showing outdated data.

There are three common ways to handle this:

1. TTL-Based Expiration

With TTL-based expiration, every cache entry has a time limit.

For example, if a profile is cached for five minutes, the system may serve stale data for up to five minutes. After that, the cache entry expires and the next request reloads it from the database.

This is simple and reliable, but it does not guarantee freshness.

2. Event-Based Invalidation

With event-based invalidation, the application deletes or updates the cache entry when the underlying data changes.

For example:

This gives fresher data, but it requires discipline. Every code path that changes the data must also handle the cache correctly.

3. Version-Based Keys

With version-based keys, the cache key includes a version number.

For example:

When the data changes, the version changes. The application starts reading from the new key, and the old cache entry is ignored until it expires or gets evicted.

This avoids explicit deletion, but it can waste cache space because old entries may stay around for some time.

Cache Stampede

A cache stampede happens when a popular cache entry expires and many requests hit the database at the same time to rebuild it.

For example, if a highly visited product page expires from cache, hundreds of requests may all miss the cache together and overload the database.

Common fixes include:

The practical rule is simple: use caching to protect your database and reduce latency, but always be clear about how stale the cached data is allowed to be.

Content Delivery Network (CDN)

A CDN is a globally distributed caching layer. Instead of everyone fetching content from your origin server, requests go to the nearest "edge" server, which might be in the same city as the user.

For example, if your origin server is in Virginia and a user in Tokyo requests an image, the request does not need to travel all the way to Virginia every time. If the Tokyo edge server already has a cached copy, it can return the image much faster.

This helps in two ways. Users get lower latency because content is served from a nearby location, and your origin server handles less traffic because repeated requests are served by the CDN.

CDNs are a good fit for static or cacheable content, such as: images, CSS, JavaScript, fonts, video segments, and sometimes public API responses with clear cache headers.

In interviews, mention a CDN when the system serves large amounts of static content, media, or globally accessed public data.

When a user places an order, several things need to happen: update inventory, send a confirmation email, notify the warehouse, update analytics, and start shipment workflows.

But not all of this needs to finish before the user sees “Order Placed.”

The critical path should stay short. The system should do only the work needed to confirm the order, then move the rest to background processing.

That is the idea behind asynchronous communication.

Instead of one service waiting for every other service to finish, it can publish work to a queue or topic and return a response quickly. Background workers handle the remaining work at their own pace.

This helps systems stay responsive under load.

Message Queues

A message queue is a buffer between producers and consumers. One service produces a message, the queue stores it, and another service consumes it when it is ready.

This decoupling is useful because the producer does not need to know who will process the message or when it will happen.

For example, the order service can place a message in a queue saying:

A worker can later pick it up and send the confirmation email.

If the email service is temporarily down, the order service does not have to fail. The message can stay in the queue until a worker is ready to process it.

When to use message queues

At the concept level, remember this: A queue decouples the producer from the speed and availability of the consumer.

Publish/Subscribe (Pub/Sub)

A queue is useful when one piece of work should be processed by one consumer.

But sometimes multiple systems need to react to the same event.

For example, when an order is placed:

This is where publish/subscribe, or Pub/Sub, helps.

In Pub/Sub, a publisher sends an event to a topic. Every subscriber to that topic receives a copy.

The order service publishes an OrderCreated event. It does not need to know which services are listening.

This makes the system easier to extend. If you later add a recommendation service or fraud detection service, it can subscribe to the same event without changing the order service.

The key idea in Pub/Sub is fan-out: one event can trigger multiple independent workflows.

Message Delivery Guarantees

Messaging systems have to deal with failure.

A producer may crash. A consumer may process a message and fail before acknowledging it. The network may drop a request. The broker itself may restart.

That is why delivery guarantees matter.

In practice, at-least-once delivery is the safest default for most designs.

You accept that duplicates can happen, then make the consumer safe to run more than once. This is called idempotency.

For example:

This is idempotent. Running it twice gives the same final result.

But this is not idempotent:

If the same message is processed twice, the user gets $200 instead of $100.

To make it safe, you usually include a unique message ID and store which messages have already been processed.

Exactly-once delivery is difficult in real distributed systems. Some systems provide strong guarantees inside their own boundaries, but the moment a message touches an external database, payment gateway, or third-party API, the guarantee becomes harder to preserve.

A practical interview answer is: Design for at-least-once delivery and make consumers idempotent.

Event Streaming

Traditional queues are usually about distributing work. A message is consumed, acknowledged, and eventually removed.

Event streaming is different.

An event stream is an append-only log. Producers append events to the log. Consumers read from it at their own pace. The events stay stored for a configured retention period.

Each consumer tracks its own position, usually called an offset.

This gives event streams a few important properties:

Replay is the big difference.

If an analytics service has a bug, you can fix the bug and replay old events. If a new service is added, it can start reading from an earlier point in the stream.

That is much harder with a simple queue, where messages are usually removed after processing.

The distinction to carry into interviews is simple: A queue distributes work. Pub/Sub broadcasts events. A stream keeps a durable history that can be replayed.

You do not need to be a networking expert for system design interviews. But you do need to understand how clients talk to servers, how requests are routed, and what options exist when the server needs to push updates back to the client.

DNS (Domain Name System)

Humans remember names like google.com. Computers connect using IP addresses.

DNS, or Domain Name System, translates a domain name into an IP address.

When you type google.com into a browser, the lookup roughly works like this:

For system design, DNS matters in a few common ways.

DNS-Based Load Balancing

A single domain can resolve to multiple IP addresses.

For example:

DNS can rotate between these IPs or return an IP based on the user's location. This helps spread traffic and route users to a nearby region.

DNS-based load balancing is useful, but it is not as precise as an application load balancer. Clients and resolvers cache DNS answers, so traffic does not move instantly when records change.

DNS can also help with failover. If one server or region becomes unhealthy, you can update DNS to point users to a healthy one.

Each DNS record has a TTL, or time-to-live. This tells clients and resolvers how long they can cache the record.

A short TTL, such as 60 seconds, helps changes take effect faster. But it also increases DNS query volume.

A longer TTL reduces DNS traffic, but failover takes longer because some clients may keep using the old IP.

A practical rule is to use shorter TTLs for records that may need quick failover, and longer TTLs for stable records that rarely change.

HTTP and REST

Most system design interviews involve APIs, and most APIs are built on HTTP.

REST is a common style for designing APIs around resources like users, orders, products, comments, and payments.

For example:

The HTTP method tells the server what kind of action the client wants to perform.

HTTP methods you need to know

Two terms matter here: safe and idempotent.

A safe request should not change server state. GET should only read data.

An idempotent request can be repeated without changing the final result. For example, sending the same PUT request twice should leave the resource in the same state.

This matters for retries.

If a network timeout happens after a PUT, the client can often retry safely. But retrying a POST may create duplicate records unless you design for idempotency, such as using an idempotency key.

HTTP Status Codes

You should also know the common status code families.

When designing APIs, be thoughtful about which codes you return. A 400 means the client sent a bad request and should fix it. A 429 means the client is sending too many requests and should slow down. A 503 usually means the server is temporarily unavailable and the client may retry later.

Real-Time Communication

Standard HTTP follows a request-response model.

The client asks. The server replies. That works well for most APIs. But some systems need the server to push updates to the client.

Examples include: Chat messages, Live sports scores, Multiplayer games, Stock prices, Collaborative editing etc..

There are a few common approaches.

Polling

With polling, the client repeatedly asks the server for updates.

Polling is simple and easy to scale, but it is inefficient.

If the client polls every five seconds, updates may be delayed by almost five seconds. If many clients poll frequently, the server handles a lot of requests even when nothing has changed.

Polling is fine for simple systems or infrequent updates.

Long Polling

Long polling improves on polling.

The client sends a request, and the server holds it open until there is an update or a timeout. Once the client gets a response, it immediately sends another request.

This reduces unnecessary empty responses and gives lower latency than normal polling.

The trade-off is that the server now keeps many requests open, which requires more careful connection management.

WebSockets

WebSockets create a persistent, two-way connection between the client and server.

Once connected, both sides can send messages at any time.

WebSockets are a good fit for systems that need low-latency, bidirectional communication, such as chat apps, multiplayer games, trading platforms, and collaborative tools.

The downside is that WebSocket connections are stateful. Scaling them is harder than scaling plain HTTP requests.

You need to think about:

Server-Sent Events

Server-Sent Events, or SSE, allow the server to push updates to the client over a long-lived HTTP connection.

Unlike WebSockets, SSE is one-way. The server can send updates to the client, but the client cannot send messages back on the same connection.

SSE is simpler than WebSockets and works well for live feeds, notifications, dashboards, logs, and score updates.

The practical rule is simple: Use normal HTTP for request-response APIs. Use polling when updates are rare and simplicity matters. Use SSE when the server only needs to push updates. Use WebSockets when both client and server need to send messages in real time.

In real systems, failure is not an edge case. It is normal.

The goal of reliability is not to prevent every failure. That is impossible. The goal is to make sure the system keeps working, or degrades safely, when parts of it fail.

A reliable system answers questions like:

Fault tolerance is the system’s ability to handle these failures without a full outage.

Redundancy

The basic rule is simple: do not depend on a single component. If something is critical, have more than one of it.

Common Examples

Redundancy does not remove failure. It gives the system another path when failure happens.

There are two common redundancy models.

Active-Active

In an active-active setup, all instances serve traffic at the same time.

If one server fails, the load balancer stops sending traffic to it and the remaining servers continue handling requests. This is efficient because all machines are used during normal operation. It is common for stateless application servers, API services, and read replicas.

The tradeoff is coordination. If the nodes share state, you need to think carefully about consistency, conflicts, and replication.

Active-Passive

In an active-passive setup, one instance handles traffic while another waits as a standby.

If the primary fails, the standby is promoted and starts serving traffic.

This model is simpler for systems where only one primary should accept writes, such as many relational database setups.

The tradeoff is that standby capacity is not fully used during normal operation. Failover also takes time, and if replication is lagging, some recent writes may be at risk.

Failover

Redundancy only helps if the system can switch to the backup when something fails. That switch is called failover.

For example, suppose a primary server is handling traffic and a backup server is waiting. If the primary fails, the system needs to detect the failure and move traffic to the backup.

A good failover design answers three questions.

How quickly can you detect the failure?

Most systems use health checks. A load balancer or monitoring service checks the primary every few seconds.

If health checks run too often, they create extra load. If they run too slowly, users see downtime for longer. The right interval depends on how quickly the system needs to recover.

How quickly can you switch traffic?

Different failover methods have different speeds.

A load balancer can stop sending traffic to a dead application server almost immediately. Database failover may take a few seconds because a replica needs to be promoted. DNS-based failover can take minutes because clients and resolvers may cache old records.

Is the backup ready?

This matters most for databases.

If the backup is a replica, it may be slightly behind the primary. This is called replication lag. If the primary fails before the replica receives the latest writes, some recent data may be lost.

The question goes beyond "Do we have a backup?" to "Can the backup safely take over right now?"

Circuit Breaker

A circuit breaker protects your system from repeatedly calling a service that is already failing.

Imagine Service A calls Service B.

If Service B becomes slow, Service A waits. Then it times out. Then it may retry. Now multiply that by thousands of requests. Soon, Service A has many threads stuck waiting for Service B. A problem in Service B starts hurting Service A too.

That is how failures spread across systems.

A circuit breaker stops this from happening. It watches calls to the downstream service. If too many calls fail, it “opens” the circuit and stops sending requests for a short time.

A circuit breaker has three states.

Closed

This is the normal state. Requests go to the downstream service.

The circuit breaker keeps track of failures. If too many requests fail within a short time, such as 5 failures in 10 seconds, it opens the circuit.

Open

In this state, requests fail fast. The service does not even try to call the downstream system.

This protects both sides. The caller does not waste threads waiting for timeouts, and the failing service gets time to recover.

Half-Open

After a short wait, such as 30 seconds, the circuit breaker allows a small number of test requests.

If the test request succeeds, the circuit closes and traffic resumes. If it fails, the circuit opens again.

The main value of a circuit breaker is fast failure.

A timeout might hold a thread for several seconds. A circuit breaker can reject the request in milliseconds. That frees the system to keep serving requests that still have a chance to succeed.

Rate Limiting

Rate limiting controls how many requests a client can make in a given time.

It protects your service from buggy clients retrying too fast, abuse or denial-of-service attempts, sudden traffic spikes, and a few users consuming too much shared capacity.

Without rate limiting, one bad client can overload the system and hurt everyone else.

Common Rate Limiting Algorithms

Token Bucket

A bucket fills with tokens at a fixed rate. Each request consumes one token.

If a token is available, the request is allowed. If the bucket is empty, the request is rejected or delayed.

Token bucket allows short bursts while still enforcing a steady long-term rate.

Sliding Window

Sliding window counts requests in a moving time range.

For example, if the limit is 100 requests per minute, the system checks how many requests the client made in the last 60 seconds. If the count is already 100, the next request is rejected.

This is more accurate than a fixed window, but usually needs more tracking.

Where to apply rate limits

Rate limits can be applied at different levels:

When a request is rate-limited, return:

As soon as you have data on more than one machine, you face a fundamental question: what happens when those machines disagree, or can't talk to each other?

CAP Theorem

CAP is useful because it asks what the system should do during a network partition:

The important point is that network partitions are not optional. Cables get cut, switches fail, and data centers lose connectivity, so a system spanning multiple nodes has to assume partitions will occur.

That makes partition tolerance a requirement rather than a choice. The real decision is only between consistency and availability when a partition is happening:

CP (Consistency + Partition Tolerance): When a partition occurs, the system may reject or block requests it cannot safely coordinate. A banking system might choose this. Better to show an error than to display the wrong account balance.

AP (Availability + Partition Tolerance): When a partition occurs, reachable nodes keep serving requests, even if some data is stale. A social media feed might choose this. Showing a slightly outdated post is better than showing nothing.

The choice depends on your use case. Ask: "What's worse, showing stale data or showing nothing?"

Treat the labels in the diagram as a starting point, not a fixed property. Several of these databases are tunable: Cassandra and DynamoDB let you raise consistency per request with quorum reads and writes, and MongoDB's behavior shifts with its read and write concern settings.

The CP/AP label describes a configuration and workload, not just the product name.

Consistency Models

Consistency isn't binary. There is a spectrum between reads that observe the latest committed write and replicas that converge later.

Strong consistency requires coordination between nodes, which adds latency. Eventual consistency is faster but means you might read stale data.

For many applications, read-your-writes consistency is a practical middle ground. Users see their own updates immediately (the experience feels consistent), but they might see other users' updates with a slight delay.

ACID vs BASE

These acronyms describe two philosophies for data management.

ACID is what traditional relational databases provide:

BASE is the typical model for distributed NoSQL databases:

ACID gives you strong guarantees but limits scalability. BASE trades some guarantees for better scalability and availability. Most real systems use a combination: ACID for transactions that must be correct (payments), BASE for data where slight staleness is acceptable (view counts).

APIs are the contract between clients and your system, and you'll design them in almost every interview. At the concepts level, four things carry most of the weight: clear resource names, correct method semantics, pagination, and idempotency.

REST Best Practices

Resource naming matters

Good APIs use nouns and hierarchy: GET /users/123, GET /users/123/orders, POST /users. Avoid verbs in paths (/getUser, /createOrder); the HTTP method already supplies the verb, so the path should name the resource.

Method semantics should be honest

GET reads and never mutates. POST creates and is neither safe nor idempotent. PUT replaces and is idempotent. DELETE removes and is idempotent.

When the method matches the behavior, clients and intermediaries (caches, proxies, retry libraries) can make safe assumptions. A GET that quietly changes state breaks all of them.

Error responses should guide the client

Return the status code that tells the caller what to do next: 400 for a malformed request the client must fix, 401/403 for auth problems, 404 for a missing resource, 429 when they should slow down, and 5xx when the failure is on your side and a retry might help.

A response body with a stable error code and a human-readable message saves the client from guessing.

Versioning keeps old clients working when you make breaking changes. URL versioning like /v1/users is explicit and easy to debug, which is why it shows up in most public APIs.

Pagination prevents returning huge result sets

Asking for "all orders" might mean ten million rows, so endpoints that list collections need a paging scheme. Two approaches dominate:

Offset pagination (LIMIT 20 OFFSET 40) is simple and lets you jump to any page, but it gets slow on deep pages because the database still scans and discards the skipped rows. Concurrent inserts or deletes can also make a row appear twice or get skipped between pages.

Cursor pagination passes a pointer to the last item seen (WHERE id > last_id) and reads forward from there. It stays fast at any depth and is stable under writes, at the cost of losing random page access. For live, frequently changing data, cursor pagination is usually the better default.

Idempotency

Network requests fail and get retried. If a payment request times out, the client often can't tell whether the charge went through, so it retries. Without protection, the customer gets charged twice.

Idempotency keys solve this. The client generates a unique key for each logical operation and includes it in the request:

The server records the key the first time it processes the request, along with the result. If the same key arrives again, it skips the work and returns the stored result, so the client can retry safely without duplicate side effects.

This matters for any operation that must not happen twice: charging a card, placing an order, or sending a notification.

One practical detail worth mentioning is that the stored key needs a time-to-live. Keeping every key forever is wasteful, and a window of 24 to 48 hours is usually enough to cover client retries.

At some point in a system design interview, the interviewer may ask:

“How much storage do we need?”

“How many requests per second should we handle?”

“How many servers would this require?”

You do not need exact numbers. You need quick, reasonable estimates that put you in the right range. The goal is to show that you can reason about capacity without getting stuck in spreadsheet-level precision.

Key Numbers to Remember

You do not need to memorize hundreds of numbers. A few rough values are enough for most interviews.

These numbers are not universal truths. They vary by hardware, network, database, and workload. But they are useful anchors during interviews.

Calculating QPS (Queries Per Second)

QPS turns a user count into a request rate, which then tells you how many servers and how much database capacity you need. This worked example takes daily active users and per-user requests down to an average and a peak QPS:

The peak multiplier matters because traffic is never evenly spread across the day. Most systems have busy hours, regional spikes, and launch-time surges. In interviews, using 2x to 5x average traffic for peak is usually reasonable unless the problem gives you better data.

Calculating Storage

Storage estimates decide whether the data fits on one machine or forces sharding and archival. This example multiplies daily record volume by record size and retention period to project growth over five years:

For interview math, round aggressively. Use 100,000 seconds per day instead of 86,400. Use 400 days instead of 365 if it makes the arithmetic easier. The answer will still be in the right order of magnitude.

Calculating Bandwidth

Bandwidth shows whether a single network link or load balancer can move the response data your traffic generates. This calculation multiplies request rate by average response size and converts the result to bits per second:

This tells you whether a single machine, load balancer, or network link is enough, or whether you need to spread traffic across more machines and regions.

These calculations help you answer questions like "do we need to shard?" or "how many servers?" If your math shows 1 million QPS and each server handles 1,000 QPS, you need roughly 1,000 servers.

The exact number is less important than the thinking. A good estimate gives you enough confidence to say, “This will not fit on one machine,” or “This database table will grow too large,” or “We need caching on the read path.”

When you're in an interview, this table helps you quickly identify which concepts apply:

These concepts show up in almost every system design interview. If you understand them well, you will have the vocabulary and mental models to reason through most design problems.

Start simple, then scale

Begin with the simplest design that can work. Add complexity only when you have a reason: more traffic, stricter latency, higher availability, or larger data volume.

Every choice is a tradeoff

There is no perfect database or one correct architecture. SQL gives strong consistency and powerful queries, but may need extra work to scale writes. Caching improves read latency, but makes freshness harder. Good design is about choosing the right trade-off for the problem.

Think about failures

Servers crash, networks split, queues back up, and databases become slow. For every major component, ask: what happens if this fails, and how does the system recover? Interviewers often ask this directly, so it helps to think through it before they do.

Use numbers to guide decisions

Back-of-envelope calculations turn a vague design into a concrete one. Saying “we need sharding” is just a claim. Saying “we have 10 million writes per day, which is roughly 115 writes per second, so a single PostgreSQL primary can likely handle the initial load” shows real reasoning.

This chapter covered the core concepts you need to know for interviews: caching, replication, sharding, queues, and consistency. In the next chapter, we will map these concepts to the technologies engineers use, and learn how to justify those choices in an interview.