Premium Video

This video is available to premium subscribers only

Unlock Full Access

HTTP is one of the foundations of modern web communication. Whether you're browsing a website, calling an API, using a mobile app, or communicating between backend services, there's a good chance HTTP is involved.

In this chapter, we'll explore how HTTP works, what happens when a client sends a request and receives a response, and why sending data over plain HTTP can be insecure.

HTTP stands for HyperText Transfer Protocol. It's the application-layer protocol that browsers, mobile apps, and backend services use to talk to web servers and APIs.

HTTP follows a simple request-response model. The client sends a request asking for something, and the server sends back a response.

When you open a web page, your browser sends an HTTP request for the page. The server responds with the HTML, and then the browser sends more requests for images, scripts, and stylesheets.

HTTP doesn't move bytes across the network by itself. It relies on a transport protocol underneath:

By default, plain HTTP uses port 80, and HTTPS uses port 443.

Before the first request is sent, a few other steps usually happen. The client looks up the server's IP address through DNS, opens a connection, and for HTTPS, sets up TLS:

So, what does an HTTP request look like?

In HTTP/1.1, a request is sent in a text-based format. Here is a simple example:

The first line is called the request line. It contains three things:

Next come the headers, which provide additional information about the request:

Finally, there's an optional request body, which contains the data being sent to the server. In this example, it holds the order details.

A GET request, on the other hand, typically doesn't have a body because it's used to retrieve data rather than send new data to the server:

HTTP/2 and HTTP/3 don't send this exact text on the wire. They use binary frames instead. But the same parts are still there: method, path, headers, and body.

The response follows a similar structure:

The first line is the status line, which includes the HTTP version and a status code. Status codes are grouped into five ranges:

The difference between 4xx and 5xx matters in practice:

429 Too Many Requests is a useful exception to remember. It is a client error, but retrying after a delay is the expected response, often guided by a Retry-After header.

Use status codes as part of the API contract. Returning 200 OK with an error object in the body confuses caches, metrics, SDKs, and retry logic, because they all read the status code to decide what happened.

Response headers then describe the result, for example how the client can cache it, and the body carries the actual content.

Caching Headers

Cache-Control: max-age=60 tells the client it can reuse this response for 60 seconds without asking again. After that, the client can ask whether the content has changed instead of downloading it again:

The ETag is a version tag for the response. A 304 Not Modified means the cached copy is still current, so no body is sent. If the response depends on a request header, such as Accept-Language, the server sends a Vary header naming it. That tells caches to store a separate copy for each value, so a French page isn't served to an English reader.

For personalized or sensitive responses, use Cache-Control: private or no-store so shared caches don't keep them.

The HTTP method tells the server what action the client wants to perform. The most common methods are:

Two properties are especially important in distributed systems: safety and idempotency.

A method is safe if it doesn't modify the server's state. GET is an example.

A method is idempotent if sending the same request multiple times has the same effect as sending it once. GET, PUT, and DELETE are idempotent. For example, deleting order 9001 twice still leaves it deleted.

POST, however, is not idempotent by default. If a client sends the same request to create an order twice, the server might create two separate orders.

This matters when a request times out. The client doesn't know whether the server processed it, so it may retry. Retrying an idempotent request is generally safe, but retrying a POST could charge a customer twice.

To prevent this, payment APIs often use idempotency keys. The client includes a unique key with each POST request, allowing the server to recognize retries and return the original result instead of processing the same operation again.

The server stores the key with the result of the first request. If the same key arrives again, it returns the stored result instead of charging again.

HTTP is a stateless protocol, meaning each request is handled independently. The server doesn't automatically remember anything about previous requests from the same client.

This might sound like a limitation, but it actually makes HTTP easier to scale. Since requests don't depend on state stored by a particular server, a load balancer can distribute them across multiple servers.

Of course, most applications still need to identify their users. They typically do this by having the client include a cookie or an authentication token with each request, often in the Authorization header. The server uses this information to identify the user and process the request.

If additional session information is needed, it can retrieve it from a shared data store, such as Redis or a database.

So the protocol is stateless, but the system around it is not. The useful question when designing an API is where that state lives, and what happens when a request is retried or lands on a different server.

But there's an important security concern we haven't addressed yet. With plain HTTP, requests and responses are transmitted without encryption, meaning anyone who intercepts the traffic can potentially read their contents.

When you send an HTTP request, it doesn't go directly to the server. It passes through your Wi-Fi router, your internet provider, and several other networks before reaching its destination.

With plain HTTP, anyone who can intercept the traffic along that path can potentially read or modify it. This creates three major security risks:

HTTPS addresses these three problems by encrypting the connection, protecting data from modification, and verifying the server's identity.

HTTPS isn't a completely different protocol from HTTP. It's simply HTTP running over a secure connection established by TLS, which stands for Transport Layer Security.

TLS replaced an older protocol called SSL, which is why you still hear the term "SSL certificate." SSL itself is outdated and shouldn't be used.

TLS provides three important security guarantees, each addressing one of the problems we just discussed:

The HTTP methods, headers, status codes, and message bodies remain the same. HTTPS simply protects their transmission using TLS.

What HTTPS Does Not Do

HTTPS protects data while it moves between two points. It does not solve every security problem:

But before the client and server can exchange encrypted data, they need to establish a secure connection and agree on the encryption keys. This happens during the TLS handshake.

Let's see how this works in TLS 1.3.

At this point, the secure connection is established, and HTTP requests and responses can be encrypted using symmetric keys derived from the shared secret. Symmetric encryption is much faster than public-key cryptography, which is why it is used for the actual data.

How the Client Checks the Certificate

A certificate binds a domain name to a public key. It is signed by a Certificate Authority (CA), an organization that browsers and operating systems already trust.

When the client receives the certificate, it checks that:

The signature in step 5 then proves the server actually holds the matching private key. A stolen copy of the certificate alone is not enough to impersonate the server.

Because the key shares are created fresh for each connection, TLS 1.3 also gives forward secrecy: if the server's private key is stolen later, previously recorded traffic still can't be decrypted with it.

Round Trips

In TLS 1.3, the handshake takes just one round trip, compared to two in a typical TLS 1.2 handshake.

For a new HTTPS connection over TCP, that generally means one round trip to establish TCP and another for TLS before the client can send its first HTTP request.

Early versions of HTTP typically opened a new TCP connection for each request. With HTTPS, this could also mean repeating the TCP and TLS handshakes for every request.

HTTP/1.1

HTTP/1.1 improved this by introducing persistent connections, also called keep-alive. Instead of opening a new connection each time, the client could reuse the same connection for multiple requests.

But there was still a limitation. In practice, requests on the same connection were handled one at a time. If one response was slow, the requests behind it had to wait. This is called head-of-line blocking.

Browsers worked around it by opening multiple connections to the same domain, typically around six.

HTTP/2

HTTP/2 addressed this problem by changing how data is transmitted. It introduced a binary format and multiplexing, allowing multiple requests and responses to share a single connection simultaneously.

Data is divided into smaller frames that can be interleaved, so one slow response doesn't have to block the others.

HTTP/2 also introduced header compression (HPACK), reducing the overhead of repeatedly sending the same headers, such as cookies and authorization tokens.

However, HTTP/2 still runs over TCP, which delivers data in order. If a packet is lost, TCP holds back subsequent data until that packet is retransmitted, even if the data belongs to a completely different request.

So HTTP/2 solved head-of-line blocking at the HTTP layer, but the problem still existed at the TCP layer.

HTTP/3

HTTP/3 addresses this by replacing TCP with QUIC, a transport protocol built on top of UDP.

QUIC handles reliable delivery independently for each stream. If a packet is lost, only the streams affected by that packet need to wait. Other streams can continue without being blocked.

QUIC also integrates TLS 1.3 directly into its handshake. This allows a new secure connection to be established in just one round trip, compared to the two typically needed for TCP and TLS 1.3 combined.

And when reconnecting to a server, QUIC can sometimes send application data immediately using 0-RTT resumption, reducing latency even further. The tradeoff is that 0-RTT data can be replayed by an attacker, so servers should only accept it for requests that are safe to repeat, such as GET.

QUIC connections can also survive a network change, such as a phone moving from Wi-Fi to mobile data, because a connection is identified by an ID rather than by IP address and port.

HTTP/3 isn't always faster. Some networks block or slow down UDP, so clients and servers keep HTTP/2 as a fallback.

So how does all of this apply when you're designing a system?

TLS Termination

In most architectures, clients don't connect directly to application servers. Instead, they connect through a load balancer, reverse proxy, or CDN, which handles the TLS connection. This is called TLS termination.

This component decrypts the incoming traffic, examines the HTTP request, and routes it to the appropriate backend server.

This centralizes certificate management and reduces the encryption work handled by application servers. It also allows the load balancer to make routing decisions based on HTTP paths and headers, which it couldn't inspect without decrypting the traffic.

What happens next depends on the system's security requirements:

Since the backend no longer sees the client's connection directly, the load balancer usually passes the original details in headers, such as X-Forwarded-For for the client IP and X-Forwarded-Proto for the original protocol. Only trust these headers when your own infrastructure sets them, since a client can send fake values.

Connection Reuse

Establishing a new HTTPS connection requires additional round trips before the client can exchange application data. For users connecting to servers on another continent, a single round trip can take over a hundred milliseconds.

That's why applications use persistent connections and connection pooling to avoid repeating handshakes unnecessarily. An HTTP client that opens a fresh connection for every call to another service pays the TCP and TLS cost every time.

It's also one reason CDNs are valuable. By terminating TLS at edge locations closer to users, they reduce network latency and make connection establishment faster.

Certificate Management

TLS certificates have expiration dates, and if a certificate expires, clients may reject connections to your service. Expired certificates are a common cause of avoidable outages.

To avoid this, teams typically automate certificate issuance and renewal. Services like Let's Encrypt use a protocol called ACME to automate this process, reducing the risk of outages caused by expired certificates. Cloud services such as AWS Certificate Manager handle renewal for certificates attached to their load balancers.

It also helps to monitor certificate expiry dates and alert well before they arrive, especially for certificates managed by hand.

Timeouts and Retries

Every HTTP client call to another service should have a timeout. Without one, a single slow dependency can tie up threads and connections until the caller fails too.

When retrying, follow the method rules from earlier. Retry idempotent requests, use idempotency keys for POST, and add backoff with some randomness (jitter) so that many clients don't retry at the same moment.

HTTP is a request-response protocol. A request has a method, a path, headers, and an optional body. A response has a status code, headers, and a body. Status codes tell the client whether to fix the request (4xx) or try again later (5xx).

Method behavior matters for retries. GET, PUT, and DELETE are idempotent, while POST needs an idempotency key to be retried safely.

HTTP is stateless, which makes it easy to spread requests across servers. Cookies, tokens, and shared stores carry the state.

Plain HTTP can be read, changed, or faked by anyone on the path. HTTPS runs HTTP over TLS, which adds encryption, integrity, and server authentication through certificates. The TLS 1.3 handshake takes one round trip.

HTTP/1.1 added persistent connections, HTTP/2 added multiplexing over one TCP connection, and HTTP/3 moved to QUIC to remove TCP-level head-of-line blocking.

So when you're designing a system, HTTPS is generally used between the client and the edge. The key decisions are where TLS terminates, how traffic is protected between internal services, and how to minimize the overhead of establishing new connections.