Every system design interview touches networking. Whether you are designing a chat application, URL shortener, or video streaming platform, data has to move between clients, services, regions, and storage systems.

Understanding networking fundamentals helps you make better design decisions and explain your choices to interviewers.

This chapter focuses on the networking concepts that matter in interviews: how data moves through the stack, when TCP or UDP matters, how DNS and HTTP affect latency, how TLS protects traffic, and where distributed systems run into network limits.

When you click a link in your browser, dozens of things happen before the page appears. Your computer looks up the IP address, establishes a connection, negotiates encryption, sends the request, and receives the response.

Each step involves a different protocol, and these protocols are organized into layers.

Why layers?

Because networking is complex, and we need a way to manage that complexity. Each layer handles one concern, whether that is routing packets across the internet or ensuring data arrives without corruption.

1.1 The OSI Model

The OSI (Open Systems Interconnection) model divides networking into 7 conceptual layers. It is a teaching model, but it gives you useful vocabulary for debugging.

Why 7 layers?

Each layer abstracts complexity from the layers above it.

When you write an HTTP request, you do not think about packet routing or electrical signals. The layers below handle those concerns, while the application layer focuses on request semantics.

More importantly, understanding layers helps you debug problems. If your service cannot reach a database, is it name resolution, a firewall blocking the port, a TLS handshake failure, or a routing problem?

Knowing the stack helps you ask the right questions.

1.2 The TCP/IP Model

The OSI model is a teaching tool. The TCP/IP model maps more directly to how the internet is implemented. It combines several OSI concepts into fewer practical layers.

1.3 Data Encapsulation

When your application sends data, it does not go directly onto the wire. Each layer adds the metadata it needs, such as ports, IP addresses, and link-layer framing. This is encapsulation.

Every device that communicates at the IP layer needs an address. IP addresses identify where packets should be delivered.

2.1 IPv4 vs IPv6

When the internet was designed in the 1980s, 4.3 billion addresses seemed like plenty. With smartphones, IoT devices, and cloud services, we ran out of IPv4 addresses years ago.

IPv4 uses 32-bit addresses like 192.168.1.1. These addresses are scarce, so we resort to tricks like NAT (Network Address Translation) to share them.

IPv6 solves this with 128-bit addresses like 2001:db8::1. The address space is enormous, which reduces the need for address sharing through NAT. Adoption is uneven: common in mobile and some cloud networks, but many internal systems still use IPv4.

For many interview designs, IPv4 is still the default assumption. Mention IPv6 when discussing mobile networks, dual-stack public endpoints, very large address spaces, or environments where NAT avoidance matters.

2.2 Public vs Private IP Addresses

Some addresses work only within private networks. Others are routable across the public internet.

Database servers should usually have private addresses and no direct inbound route from the public internet. A public load balancer or edge proxy receives user traffic and forwards it to private application tiers. Getting this wrong creates either security exposure or connectivity problems.

NAT (Network Address Translation) allows multiple devices with private IPs to share a single public IP. Your home router performs NAT.

2.3 Subnetting and CIDR

When you design a system on AWS, GCP, or any cloud provider, one of the first decisions is how to carve up your network. CIDR notation helps you specify IP ranges concisely.

The notation 192.168.1.0/24 means "the first 24 bits are the network, the remaining 8 bits are for hosts." In classic IPv4 subnetting, that gives 256 addresses, with 254 usable because one is the network address and one is broadcast. Cloud providers may reserve additional addresses in each subnet.

The smaller the number after the slash, the larger the network:

Why bother with subnets?

Subnets let you segment your network for security and control. A typical production setup might look like this:

The public subnet holds your load balancer, which has a public IP. The private subnet holds your application servers, which are not directly reachable from the internet.

The data subnet holds your database, isolated from direct internet access and reachable only from approved application tiers. Subnets are not a security boundary by themselves; enforce the boundary with security groups, firewall rules, network ACLs, and identity-aware controls where available.

2.4 How Routing Works

IP addresses tell us where to send data, but they do not tell us how to get there. That is the job of routing. Every router along the path looks at the destination IP and decides which direction to forward the packet.

Every router has a routing table that maps destination networks to next hops:

The router checks each destination against its table, finds the most specific match (longest prefix), and forwards the packet. If no specific route matches, it uses the default route (0.0.0.0/0).

Packets may take different paths across the internet, even within the same connection. Routers make independent decisions, and conditions change. This is why TCP needs sequence numbers to reassemble data in order.

TCP is the default transport for many systems. Most HTTP/1.1 and HTTP/2 traffic, database connections, and many service-to-service calls run over TCP. It handles packet loss, reordering, and corruption detection to present a reliable ordered byte stream to the application.

3.1 TCP Connection Lifecycle

Before sending data, TCP establishes a connection. This takes a round trip, which is why connection reuse matters at scale.

The handshake accomplishes two things: verify that both sides can send and receive, and agree on initial sequence numbers.

Why random sequence numbers?

If they were predictable, an attacker could more easily inject fake packets into a connection. Random initial sequence numbers make this harder.

Why not two steps?

With only two steps, the server would not know if the client received its response. Old, delayed SYN packets could trick the server into allocating resources for connections that will never complete.

3.2 TCP Header Structure

The TCP header is at least 20 bytes long, laid out as a series of 32-bit words. Each row below is one word, and the fields inside it sit side by side from bit 0 to bit 31.

Key fields:

3.3 Reliability Mechanisms

IP is unreliable. Packets can get lost, duplicated, corrupted, or arrive out of order. TCP adds reliability on top of IP through four mechanisms that work together.

1. Acknowledgments and Retransmission

TCP acknowledges received byte ranges. When the sender does not receive progress within a timeout, or sees duplicate ACKs that imply a gap, it retransmits data.

2. Sequence Numbers

Each byte in the stream has a sequence number. If packet 3 arrives before packet 2, the receiver holds packet 3 until packet 2 shows up, then delivers both in order. If the same packet arrives twice (a retransmission that was not needed), the receiver ignores the duplicate.

3. Checksums

Every TCP segment includes a checksum computed over a pseudo-header, TCP header, and data. If the receiver's checksum does not match, TCP discards the segment. The sender retransmits if acknowledgments do not advance.

4. Sliding Window

Waiting for an ACK after every packet is slow, especially over high-latency links. The sliding window lets the sender have multiple packets "in flight" simultaneously.

3.4 Flow Control

What happens if the sender transmits faster than the receiver can process? TCP flow control prevents overwhelming the receiver by having the receiver advertise how much buffered data it can accept.

This back-pressure mechanism is automatic at the transport layer. Your application writes to the socket, and TCP paces delivery based on the receiver window. But if the receiver is consistently slower than the sender, the application still needs queues, backpressure, shedding, or a different data flow.

Window Scaling for Modern Networks

The original TCP header has a 16-bit window field, limiting it to 64KB. On a 1 Gbps link with 100ms RTT, the bandwidth-delay product is about 12.5MB, so a 64KB window cannot fill the pipe. Modern TCP uses a window scaling option negotiated during the handshake to support much larger windows.

3.5 Congestion Control

Flow control prevents overwhelming the receiver. Congestion control prevents overwhelming the network.

Consider a router in the middle of the internet handling traffic from thousands of connections. If everyone sends as fast as possible, the router's queues overflow, packets get dropped, and everyone's performance suffers. TCP detects congestion and reduces its sending rate to avoid making the problem worse.

Slow Start: A new connection does not know how much bandwidth is available. It starts with a small congestion window (typically 10 segments) and doubles every round trip. This exponential growth quickly finds the available capacity.

Congestion Avoidance: Once the window hits a threshold (or after recovering from loss), growth becomes linear. Add one segment per RTT. This cautious probing avoids triggering congestion.

Fast Recovery: If the sender receives three duplicate ACKs (same acknowledgment number three times), it means a packet was lost but subsequent packets arrived. The sender retransmits immediately and halves its window, rather than starting over.

Timeout: If acknowledgments stop arriving long enough, the sender assumes significant loss or path trouble and reduces its sending rate sharply.

New TCP connections start slow. On a high-latency link (100ms RTT), it takes several round trips just to ramp up to full speed. If your system makes many short-lived connections, most of the time is spent in slow start, never reaching peak throughput.

This is why connection reuse matters. HTTP keep-alive keeps a TCP connection open after a response so the next request on the same connection skips the handshake and the slow-start ramp. Connection pooling takes this further: a client or server holds a set of warm, already-established connections and hands them out as requests arrive, instead of opening a new one each time. Database clients almost always pool connections for this reason, and HTTP/2 multiplexing and gRPC streaming reuse a single connection for many logical requests. The shared idea is to pay the setup cost once and amortize it across many requests.

Modern Congestion Control Algorithms

The classic algorithms (Reno, New Reno) use packet loss as the signal of congestion. Modern algorithms are smarter:

3.6 TCP Tuning Parameters

You rarely need to tune TCP, but when you do, these are the parameters that matter:

When You Hit TCP Limits

TCP does a lot of work: connection setup, reliability, ordering, flow control, congestion control. That work takes time and bandwidth. Sometimes you do not need it. That is where UDP comes in.

UDP strips away all the complexity. It takes your data, adds source and destination ports, and sends it. No handshakes, no acknowledgments, no retransmissions. If a packet is lost, UDP does not detect or resend it. If packets arrive out of order, UDP delivers them in that order.

4.1 UDP vs TCP

The trade-off is fundamental: reliability versus latency.

4.2 UDP Header

UDP's minimal design is visible in its wire format: the entire header is just four 2-byte fields, carrying source port, destination port, total length, and an optional checksum.

The header is 8 bytes. No sequence numbers, no acknowledgments, no connection state.

4.3 When to Use UDP

The question is not "is reliability important?" but rather "who should handle reliability?"

DNS: A DNS query is tiny (a few hundred bytes) and expects a quick response. If the response does not arrive in 2 seconds, the client asks again. TCP's handshake would take longer than the actual query. UDP fits perfectly.

Video Streaming: When you are watching a video, a lost frame causes a brief glitch but is tolerable. Waiting to retransmit it is worse because now multiple frames are stale. The video player interpolates or shows a brief glitch and moves on. UDP with application-level buffering works better than TCP here.

Online Gaming: In a multiplayer game, you send player position 60 times per second. If packet 42 is lost but packet 43 arrives, you do not want the old position. You want the latest state. TCP would deliver packet 42 first, adding latency and giving you outdated information.

Voice/Video Calls: Similar to gaming. A brief audio glitch is better than a half-second delay while TCP retransmits, and listeners can usually follow speech through small gaps.

IoT Sensors: A temperature sensor sending readings every second does not need guaranteed delivery. If one reading is lost, the next one arrives in a second anyway. UDP keeps the protocol stack minimal for constrained devices.

4.4 Building Reliability on UDP

Sometimes you want UDP's speed but need reliability for certain messages. The solution is to implement reliability at the application layer, but only where you need it.

QUIC (HTTP/3)

QUIC is the most widely deployed example of this idea. It runs over UDP but implements reliability, congestion control, encryption, and stream multiplexing in user space. The HTTP/3 section covers how it works in detail.

Game Networking

A game might have three types of messages:

The game protocol uses UDP underneath but tracks sequence numbers and acknowledgments only for the reliable messages.

Humans remember names. Computers need numbers. DNS bridges this gap, translating google.com into 142.250.80.46.

DNS is easy to overlook because it usually works. Most web requests, API calls, and email delivery paths depend on DNS. Slow DNS adds latency before the application even sees traffic; broken DNS can make a healthy service unreachable.

5.1 DNS Hierarchy

DNS is a hierarchical, distributed system spread across millions of servers worldwide, not one central database.

When you look up www.google.com, the query flows from your browser through multiple servers:

This looks slow, but caching makes it fast. Most queries hit a cache at some level and return immediately. A full recursive lookup only happens when no cached answer exists.

5.2 DNS Record Types

5.3 DNS Caching

Without caching, DNS would add avoidable latency and huge load to recursive and authoritative servers. Caching makes lookups fast, but it also creates operational trade-offs.

The TTL Dilemma

Every DNS record has a TTL (Time To Live) that controls how long it can be cached. Choosing the right TTL is a trade-off:

The complication: Caches do not always respect TTL. Some ISPs cache longer than they should. Corporate proxies add their own caching. When you change a DNS record, some users will see the old IP for longer than you expect. Plan for this during migrations.

5.4 DNS in System Design

Beyond name resolution, DNS is also a tool for traffic management.

Round-Robin Load Balancing

Return multiple A records and let clients pick one:

This spreads traffic across servers, but with significant limitations. Health behavior is coarse: some managed DNS services support health checks, but cached answers can keep sending traffic to an unhealthy endpoint until TTLs expire. Distribution is uneven, because a resolver cache may concentrate many users on the same returned address.

DNS also has no request-level session awareness, so it does not know about cookies, paths, load, or user sessions. For these reasons, DNS load balancing is usually a first layer, with a real load balancer behind it.

Geographic Routing

GeoDNS usually routes based on the resolver's IP address, sometimes helped by EDNS Client Subnet. This is useful, but it is approximate: public resolvers, VPNs, mobile networks, and corporate DNS can all make a user look like they are somewhere else.

Anycast

GeoDNS hands out different IPs to different users. Anycast does the opposite: many servers in different locations share the same IP address, and the internet's routing protocol (BGP) naturally sends each user to the topologically nearest one. The user connects to one IP, but the network decides which physical site answers.

This is how the DNS root servers and most CDNs work. It gives you proximity routing without relying on the resolver's location, and it fails over automatically: if one site withdraws its route, traffic shifts to the next nearest site. The trade-off is that routing changes can move a user mid-session, so anycast suits stateless or connectionless traffic (DNS over UDP, CDN edge requests) better than long-lived stateful connections.

Service discovery

SRV records give you the IP, port, and priority of a service:

Kubernetes and service meshes often use DNS for internal service discovery, though they may use specialized resolvers like CoreDNS rather than public DNS.

HTTP is how most web clients talk to services. It is a request-response protocol: HTTP/1.1 and HTTP/2 usually run over TCP, while HTTP/3 runs over QUIC on UDP. Knowing that distinction helps you design APIs and debug production issues without blaming the wrong layer.

6.1 HTTP Request/Response

An HTTP transaction is simple: the client sends a request, the server sends a response.

Request components:

Response components:

6.2 HTTP Methods

HTTP methods have semantic meaning. Using them correctly makes an API easier to cache, retry, and reason about.

Idempotent means repeating the same request has the same intended effect. Sending the same PUT request twice leaves the resource in the same state. This matters when the network drops and the client does not know whether the first attempt succeeded.

Safe means the request is intended only to read state. GET can still create logs or metrics, but it should not modify business data. This lets proxies cache responses and lets clients retry reads more freely.

Common mistakes

6.3 HTTP Status Codes

Status codes are grouped into five classes by their leading digit, and knowing which class a code belongs to tells you immediately whether the problem is in the client, the server, or the connection path between them.

Common status codes:

6.4 HTTP/1.1 vs HTTP/2 vs HTTP/3

Each version of HTTP addresses a different bottleneck left by its predecessor, moving from one request per connection, to multiplexed streams over TCP, to full stream independence over QUIC.

HTTP/1.1: The Problem

In common HTTP/1.1 usage, each connection has one in-flight request at a time. If you need 10 resources, you either wait or open multiple connections. Browsers typically cap parallel connections per origin, so older sites used domain sharding to spread assets across names like cdn1.example.com and cdn2.example.com.

HTTP/2: Multiplexing

HTTP/2 allows multiple requests on a single connection. The requests interleave as streams, which usually removes the need for domain sharding and reduces connection setup overhead. The protocol uses binary framing and HPACK header compression.

But HTTP/2 still runs over one TCP connection. If a TCP segment is lost, delivery of later bytes is blocked until retransmission, even if those bytes belong to another HTTP/2 stream. This is TCP-level head-of-line blocking.

HTTP/3: Moving to QUIC

HTTP/3 uses QUIC instead of TCP. QUIC performs loss recovery per stream, so packet loss on one stream does not block unrelated streams. QUIC also supports faster connection setup, connection migration across network changes, and 0-RTT resumption for replay-safe repeat requests.

6.5 HTTP Caching

Caching is often the most effective performance tool. The fastest request is the one the client can satisfy without contacting your server.

Cache-Control headers

The Cache-Control header tells browsers and CDNs what to cache and for how long:

Conditional Requests: Efficient Revalidation

When a cached response expires, the browser can ask "has this changed?" rather than re-downloading everything:

6.6 Real-Time Communication Patterns

Not every feature needs a bidirectional socket. Pick the simplest pattern that matches the direction and frequency of updates.

In interviews, call out the operational cost. WebSockets and SSE keep connections open, so you need connection-aware load balancing, heartbeats, backpressure, and a plan for reconnects. For low-frequency updates, polling is often simpler and good enough.

HTTPS is HTTP over TLS. Without TLS, anyone on the network path, your ISP, a coffee shop router, or a compromised router, can read or modify traffic. With TLS, they see encrypted bytes and limited connection metadata.

TLS provides three things:

7.1 TLS Handshake

Before encrypted communication begins, client and server must agree on encryption keys. This is the TLS handshake.

TLS 1.3 improvements

TLS 1.2 requires two round trips. TLS 1.3 cuts this to one.

TLS 1.3 also removes outdated cipher suites, encrypts more of the handshake, and supports 0-RTT resumption for repeat visitors. It does not hide everything by itself: the Server Name Indication (SNI) is still visible unless Encrypted Client Hello is used and supported.

7.2 Certificate Chain

How do you know you are talking to google.com and not an attacker? Certificates.

Your browser trusts a set of Certificate Authorities (CAs). When a server presents its certificate, the browser verifies it was signed by a trusted CA.

What the browser checks

If any check fails, the browser shows a warning. Never train users to click through these warnings.

7.3 TLS Termination Strategies

Where do you decrypt HTTPS traffic? This is a key architectural decision.

For many applications, edge termination at the load balancer is enough when internal traffic is isolated with VPCs, security groups, private subnets, and tight IAM. For regulated, multi-tenant, or zero-trust environments, keep TLS all the way to the service and consider mTLS for service identity.

7.4 HTTPS Best Practices

Use Modern Cipher Suites

TLS 1.3 simplifies cipher selection because it only includes modern AEAD cipher suites:

For TLS 1.2, prefer ECDHE key exchange paired with AES-GCM or ChaCha20-Poly1305:

Disable anything with "CBC", "3DES", or "RC4".

Security Headers

These HTTP headers strengthen your security posture:

Automate Certificate Management

Manual certificate renewal is easy to forget and can cause avoidable outages. Let's Encrypt and other ACME providers issue free, automated certificates. AWS Certificate Manager handles rotation automatically for AWS services. HashiCorp Vault covers internal PKI and service certificates.

What users experience is responsiveness, not the architecture behind it. Responsiveness is dominated by latency, the time between an action and a visible response.

At scale, network latency often exceeds processing time. Your database query might take 5ms, but the network round trip to the user takes 100ms. Understanding where latency comes from helps you design faster systems.

8.1 Latency Components

When you send a packet across the internet, where does the time go?

8.2 Latency Numbers You Should Know

These are order-of-magnitude numbers, not constants. Hardware, cloud provider, region, network path, and load all change the exact values.

Once you leave the machine, latency jumps by orders of magnitude. If a user-facing request spends 100ms on the network, reducing a 2ms handler to 1ms helps less than reducing round trips or moving work closer to the user.

8.3 Bandwidth vs Latency

People often confuse these. They are different things, and optimizing one does not necessarily improve the other.

Bandwidth: How much data can flow per second. Think of it as pipe width.

Latency: How long it takes for the first byte to arrive. Think of it as pipe length.

For small requests (API calls, web pages): Latency dominates. Sending 10KB on a 1 Gbps link takes about 0.08ms. If latency is 100ms, the transfer time barely matters.

For large transfers (backups, video): Bandwidth dominates. Sending 1GB takes about 8 seconds on 1 Gbps, or about 80 seconds on 100 Mbps. The initial latency is a small part of the total.

For web applications serving small responses, focus on latency. For batch data pipelines, focus on bandwidth.

8.4 Reducing Latency

You cannot beat the speed of light. But you can reduce the distance it travels and the number of trips it makes.

1. Move computation closer to users

CDNs for static content, edge functions for dynamic content, and multi-region deployments for global applications.

2. Reduce round trips

Each round trip adds latency. Batch related operations when it makes the API cleaner, use HTTP/2 or HTTP/3 multiplexing where available, and keep connections alive instead of reconnecting.

3. Cache aggressively

Cache at the browser, CDN, application, or database layer when the data model allows it.

4. Compress data

Less data = less transmission time. Use gzip or Brotli for text. Use efficient binary formats (Protocol Buffers, MessagePack) for internal APIs.

5. Process asynchronously

Return a quick acknowledgment, process in the background, notify when done. The user sees a fast response even if the actual work takes time.

8.5 Tail Latency

Averages hide tail behavior. If 99% of your requests take 10ms but 1% take 5 seconds, the average looks fine while some users have a poor experience.

Why p99 matters more than average

A single page load might make 50+ requests. If an important one is slow, the page feels slow.

A page with 50 independent API calls has roughly a 39% chance of seeing at least one p99-latency call. Real systems are correlated, but the lesson holds: track tail percentiles, not just the average.

Common Causes of Tail Latency

Networking gets harder in distributed systems. Instead of one server, you have dozens or thousands. Instead of one network hop, you have many. At sufficient scale, some dependency is usually slow, unreachable, or recovering.

Design as if partial failure is normal. That mindset changes how you choose timeouts, retries, load balancing, and service discovery.

9.1 The Network is Unreliable

In a distributed system, you cannot distinguish between "the network is slow" and "the server is down." Both look the same: no response.

Dealing with failures:

9.2 Timeouts

When a request does not get a response, how long do you wait? There is no single correct timeout.

Too short: You give up on requests that would have succeeded. You trigger retries that create duplicate work. Under load, you make things worse.

Too long: Users wait forever. Resources (connections, threads) stay tied up. You detect failures slowly.

Strategies:

In practice: Keep connection timeouts short enough to fail fast for your environment, and set read timeouts based on expected operation time. For chained calls, propagate deadlines: if you have 5 seconds total and already spent 2 seconds, downstream only gets 3 seconds.

9.3 Retries and Idempotency

You sent a request. It timed out. What happened?

You cannot tell which case you are in. The solution is idempotency: design operations so that doing them twice has the same effect as doing them once.

Which Operations Are Naturally Idempotent?

Making Non-Idempotent Operations Safe

Use an idempotency key:

The client attaches a unique key to the request:

The server uses that key to deduplicate retries:

Many payment APIs use idempotency keys for this reason. With a correctly implemented key store, a retry returns the original result instead of charging the customer twice.

9.4 Service Discovery

In a dynamic environment, IP addresses change. Servers come and go. How does Service A find Service B?

Most cloud environments use a combination. Kubernetes uses DNS (CoreDNS) for service names, while service meshes such as Istio or Linkerd add mTLS, telemetry, retries, and traffic policy. The registry or control plane must stay fresh; stale endpoints are a common source of flaky calls.

9.5 Load Balancing in Distributed Systems

Load balancers distribute traffic across servers. The choice of algorithm affects performance and reliability. A load balancer is one example of a reverse proxy, so it helps to be clear on the two kinds of proxy.

Forward vs Reverse Proxy

A forward proxy sits in front of clients and makes requests on their behalf. The server sees the proxy, not the original client. Corporate web filters and outbound gateways are forward proxies.

A reverse proxy sits in front of servers and receives requests on their behalf. The client sees the proxy, not the backend. Load balancers, API gateways, TLS-terminating edges, and CDNs are reverse proxies. They are where you centralize TLS termination, routing, caching, rate limiting, and health checking, which is why most production traffic enters through one.

Layer 4 vs Layer 7

Load balancing algorithms

Server-Side vs Client-Side

Here are the key takeaways: