AlgoMaster Logo

TLS/SSL Deep Dive

High Priority26 min readUpdated August 14, 2026
Listen to this chapter
Unlock Audio

When an application sends data across a network, every device on the path handles some part of that traffic. Without additional protection, a nearby attacker, an untrusted network, or a compromised intermediary may be able to read or alter the application bytes.

Transport Layer Security (TLS) creates a protected channel between two endpoints. HTTPS is HTTP carried through TLS, but TLS is not limited to the web. Database clients, email protocols, message brokers, service-to-service APIs, and many other systems use it.

A TLS connection has two broad phases:

  1. The handshake negotiates parameters, authenticates one or both peers, and establishes shared secrets.
  2. The record protocol uses keys derived by the handshake to protect application data.

Modern systems should use TLS terminology even though people still say “SSL certificate” or “SSL connection.” Secure Sockets Layer, or SSL, was TLS's predecessor. SSL 2.0 and SSL 3.0 are obsolete and must not be used. This chapter focuses on TLS 1.3, with enough TLS 1.2 context to understand older deployments.

What TLS Provides

A correctly established TLS connection provides three central security properties.

Confidentiality means a network observer cannot directly read the protected application data.

Integrity means an attacker cannot modify, insert, or remove protected bytes without detection.

Authentication means an endpoint can establish the identity of its peer. In the common web model, the client authenticates the server using a certificate. TLS can also authenticate the client.

These properties apply to a particular TLS connection between particular endpoints. They do not automatically make the entire application secure.

Suppose a client sends an HTTPS request to a reverse proxy, which forwards it to an application server:

The first TLS connection ends at the proxy. The proxy receives plaintext and can inspect it. If the proxy-to-backend hop must also be protected, it needs a separate secure connection.

TLS also does not decide whether an authenticated user may access an account, prevent a server from storing passwords insecurely, or stop malware running inside an endpoint. It protects data in transit between its endpoints.

Where TLS Fits

For a typical HTTPS connection over TCP, the layers appear in this order:

The client first establishes a TCP connection. TLS then exchanges handshake messages through that byte stream. Only after the secure channel is ready does the application normally send HTTP data.

TLS does not replace TCP reliability. A TLS implementation expects the underlying stream to deliver bytes in order without gaps. TCP retransmits lost segments; TLS verifies the authenticity of the bytes after they arrive.

TLS can also be integrated into transports other than this classic stack. HTTP/3 uses the TLS 1.3 handshake inside QUIC, for example, but QUIC protects packets using its own record-like mechanisms rather than placing the ordinary TLS record layer over TCP. The security ideas remain similar even though the wire layout changes.

The Cryptographic Jobs Inside TLS

TLS combines several cryptographic mechanisms because no single mechanism efficiently solves every problem.

Ephemeral Key Agreement

The client and server need to establish a shared secret even though their initial messages cross an untrusted network. TLS 1.3 commonly uses ephemeral Elliptic Curve Diffie–Hellman, written ECDHE, for this job.

Each endpoint creates a temporary private value and a corresponding public key share:

The client combines a with B, while the server combines b with A. The mathematics produces the same shared secret on both sides. An observer sees A and B but cannot feasibly recover that secret.

These key shares do not authenticate the endpoints by themselves. An active attacker could otherwise establish one secret with the client and another with the server. TLS binds the key agreement to authenticated handshake messages to prevent that substitution.

Because the private values are temporary, later compromise of the server's long-term certificate private key does not reveal the ephemeral shared secret from an old connection, assuming the temporary secrets and traffic keys have been erased. This property is called forward secrecy.

Digital Signatures and Certificates

A server certificate associates a public key with an identity. The certificate is normally part of a chain leading to a trust anchor already known to the client.

During a TLS 1.3 handshake, the server signs the handshake transcript using the private key corresponding to its certificate. This signature proves possession of that private key and binds the server's identity to this specific exchange.

Receiving a certificate is not enough on its own. The client must also decide whether the certificate is acceptable for the intended service, and it must verify the server's signature. Certificate validation has several rules and failure modes; here, the important point is that it supplies the authentication needed by the handshake.

Symmetric Authenticated Encryption

Public-key operations are useful during setup but unnecessarily expensive for every application byte. After the handshake, TLS uses fast symmetric keys known to both endpoints.

TLS 1.3 cipher suites use Authenticated Encryption with Associated Data (AEAD) algorithms. An AEAD operation produces:

  • Ciphertext that hides the plaintext
  • An authentication tag that detects modification
  • Authentication for selected unencrypted metadata, called associated data

AES-GCM and ChaCha20-Poly1305 are common AEAD algorithms. If record authentication fails, the receiver terminates the connection rather than returning partially trusted data.

Hashes and HKDF

TLS continually hashes the handshake messages into a transcript hash. It also uses the HMAC-based Extract-and-Expand Key Derivation Function, or HKDF, to turn shared secret material into multiple independent secrets.

This design gives TLS different keys for different purposes and directions. Client-to-server handshake traffic, server-to-client handshake traffic, client-to-server application traffic, and server-to-client application traffic do not all reuse one key.

The TLS 1.3 Full Handshake

Consider a new client connecting to api.example.com with no reusable session. A normal server-authenticated TLS 1.3 handshake takes one network round trip before the client can send protected application data:

Both derive handshake traffic keysAuthenticate server and verify transcriptApplication traffic keys are activeClientHelloversions, cipher suites, key share, extensionsServerHelloselected parameters and key shareEncryptedExtensionsCertificateCertificateVerifyFinishedFinishedEncrypted application dataEncrypted application dataClientServer
12 / 12
algomaster.io

The diagram shows handshake messages, not a one-to-one mapping to TCP segments or TLS records. Several messages can share one record, and TCP can split a record across several segments.

Step 1: ClientHello

The client begins by describing what it supports. A TLS 1.3 ClientHello commonly includes:

The exact offers depend on the TLS library and application.

The Server Name Indication (SNI) extension tells a server which hostname the client wants. It is needed because one IP address can serve certificates for many domains.

The Application-Layer Protocol Negotiation (ALPN) extension lets the peers select the protocol that will run inside TLS. An HTTPS client might offer HTTP/2 as h2 and HTTP/1.1 as http/1.1.

The client sends a key share immediately so that the server can usually finish negotiation without an extra exchange. If the server supports one of the client's groups but needs a different key share, it can send HelloRetryRequest. The client then sends a revised ClientHello, costing another round trip.

Some fields retain legacy-looking values for compatibility with old network devices. For example, a TLS 1.3 client advertises its real versions in supported_versions; a field literally named legacy_version does not provide the final negotiated version. Packet analysis must interpret the extensions instead of trusting that old field.

A packet capture may also show a ChangeCipherSpec record during a TLS 1.3 handshake. In compatibility mode, this is a dummy record that the peer ignores. The real key transitions are determined by the TLS 1.3 handshake state, not by that legacy-looking record.

Step 2: ServerHello

The server chooses parameters from the client's compatible offers. Its ServerHello identifies:

  • TLS 1.3 as the selected version
  • One TLS 1.3 cipher suite
  • A compatible key-exchange group
  • The server's ephemeral public key share
  • A fresh server random value

At this point, both endpoints can compute the ECDHE shared secret. They combine it with the transcript so far and run it through the TLS 1.3 key schedule. The result includes separate client and server handshake traffic secrets.

Messages after ServerHello are encrypted with keys derived from those secrets. A passive observer can normally read the initial ClientHello and ServerHello, but it cannot read the server's following certificate or most other handshake details.

The selected cipher suite might be:

In TLS 1.3, this name selects AES-128-GCM for record protection and SHA-256 for hashing and HKDF. It does not select the certificate type or the key-exchange group. Those are negotiated independently through extensions.

This separation matters because TLS 1.2 cipher suite names bundled many choices together:

Reading a TLS 1.3 cipher suite as if it encoded all four decisions leads to incorrect conclusions.

Step 3: EncryptedExtensions

EncryptedExtensions carries the server's responses to extension offers that are not needed to establish the initial cryptographic context.

For HTTPS, one important result is the selected ALPN protocol. If the client offered h2 and http/1.1, the server may select:

The client now knows how to interpret the application bytes that will follow. The selection is encrypted and bound to the handshake transcript, so it cannot be changed silently.

Step 4: Certificate

In the common certificate-based handshake, the server sends its certificate chain. The first certificate describes the server, and following certificates normally help connect it to a trusted issuer.

Conceptually, the client needs to establish that:

  • The chain leads to an acceptable trust anchor.
  • The certificate is valid for the intended service identity.
  • The certificate is currently usable for server authentication.
  • The signatures and relevant constraints are valid.

Those checks answer whether the public key should be trusted for the connection. They are distinct from checking the next message, which proves that the current server actually controls the matching private key.

Step 5: CertificateVerify

The server signs a value derived from the handshake transcript with its certificate private key. The client verifies the signature using the public key from the server certificate.

This step connects three facts:

A copied certificate is public information and proves nothing by itself. The private-key signature is the proof of possession.

Step 6: Server Finished

The server derives a special finished_key from its handshake traffic secret. It uses that key to calculate a message authentication code over the transcript hash.

The client independently calculates the expected value. A match demonstrates that the server derived the same handshake secrets and that the transcript it authenticated matches the client's view.

This protects the complete negotiation. An attacker cannot silently remove an offered version, substitute a key share, or alter an ALPN choice and still produce a valid Finished message.

Once the server sends Finished, it can derive server application traffic keys and may send application data. The client still verifies the server's authentication and the server Finished before accepting that data.

Step 7: Client Finished

After verifying the server's certificate, signature, and Finished, the client sends its own encrypted Finished message. It is computed from the client handshake traffic secret and the transcript.

The server verifies it to confirm that the client saw the same handshake and derived the same secrets. The client can then send ordinary protected application data.

If the server requested client authentication, the client's final flight also includes Certificate and CertificateVerify before Finished. This is the basis of mutual TLS.

Why the Transcript Matters

The handshake is not a collection of independent messages. Each important result depends on the ordered bytes exchanged so far.

Imagine an attacker changes the client's ALPN offer from:

to:

The client hashes the original ClientHello; the server hashes the modified one. Their transcript hashes differ. Values derived from or signed over the transcript will no longer verify, so the handshake fails.

This principle protects much more than ALPN. Version negotiation, algorithms, identities, key shares, extensions, and previous handshake messages are cryptographically bound together.

The transcript also prevents messages from one handshake being freely transplanted into another. Fresh random values and ephemeral key shares make each full handshake distinct.

The TLS 1.3 Key Schedule

Saying that TLS “creates a session key” hides an important design. TLS 1.3 creates a hierarchy of secrets, each scoped to a phase and purpose.

The diagram is conceptual rather than a literal list of every HKDF operation.

The Early Secret accounts for an optional pre-shared key. In a full handshake without one, TLS uses a defined zero input in its place.

The Handshake Secret mixes in the ECDHE result. Client and server handshake traffic secrets derived from it protect the encrypted portion of the handshake.

The Master Secret is the root for application traffic secrets, exporter secrets, and resumption material. The application traffic secrets produce the actual write keys and initialization vectors used by the record layer.

Each direction has independent secrets:

An endpoint uses its write key to encrypt outgoing records and its peer's write key to decrypt incoming records. Independent keys limit accidental cross-direction reuse and make key updates possible.

TLS can derive later generations of application traffic keys using KeyUpdate. Once an implementation replaces and erases an old secret, compromise of the new secret does not reveal records protected by the erased generation. Key updates do not repair an already compromised live connection or authenticate an endpoint again; they rotate record-protection keys within the existing connection.

The Record Protocol

After the handshake derives traffic keys, the record protocol protects a stream of handshake, alert, and application bytes.

A TLS record is a framing unit. It is not an HTTP request, a database query, a TCP segment, or an application message. One application message can span multiple records, and one record can contain bytes from multiple application writes.

TLS 1.3 plaintext fragments are at most 2^14 bytes, or 16 KiB, before protection. At a high level, a protected record contains:

The outer header carries:

Once protection is active, the outer content type normally looks like application_data even when the encrypted inner content is a handshake or alert message. The real content type is placed inside the ciphertext. This hides more protocol detail from observers.

Nonces and Sequence Numbers

AEAD encryption requires a nonce that must not repeat under the same key. TLS 1.3 maintains a separate 64-bit record sequence number for each traffic direction and key generation.

For each record, TLS combines the current sequence number with the direction's static write IV to construct the nonce. The sequence number begins at zero when a traffic key becomes active and increments for every record.

The sequence number is implicit rather than transmitted as a separate TLS 1.3 record field. Both endpoints infer it from record order. TCP's reliable, ordered stream ensures that the receiver does not skip an encrypted record and then unknowingly use the wrong nonce for the next one.

Changing a visible header, ciphertext byte, or authentication tag causes AEAD verification to fail. The receiver treats that as a fatal connection error; it must not try to continue past corrupted protected data.

Padding and Metadata

TLS 1.3 permits zero-byte padding inside an encrypted record. Padding can make different plaintext lengths produce the same visible record length, but most traffic is not padded enough to hide its full behavior.

An observer can still see:

  • Source and destination IP addresses
  • Transport ports
  • Connection timing and duration
  • Packet and record sizes
  • Direction and volume of traffic

TLS protects content, not all metadata. Traffic patterns may still reveal useful information even when decryption is impossible.

Loading simulation...

What Is Visible During the Handshake

In an ordinary TLS 1.3 handshake, the initial ClientHello and ServerHello are not protected by the handshake traffic keys. A network observer can therefore see offered parameters and often the requested server name in SNI.

Most later handshake messages are encrypted, including the server certificate. This is an improvement over TLS 1.2, where more of the handshake is visible.

Encrypted Client Hello (ECH) is a TLS 1.3 extension designed to protect sensitive ClientHello contents, including the true SNI. With ECH, a client encrypts an inner ClientHello using configuration obtained for the service and sends a less-sensitive outer ClientHello for routing and compatibility.

ECH reduces hostname leakage when both client and service support it and the client obtains trustworthy configuration. It does not hide destination IP addresses, sizes, timing, or every outer routing detail. TLS 1.3 should therefore not be described as automatically hiding the requested hostname; that depends on ECH use and the surrounding name-resolution path.

Loading simulation...

Session Resumption

A full handshake performs fresh key agreement and normally sends the server certificate chain. Repeating all of that for every short connection adds latency and CPU work.

After a successful TLS 1.3 connection, the server can send one or more NewSessionTicket messages. Each ticket represents a pre-shared key, or PSK, derived from the completed handshake. The client stores the ticket together with the associated resumption secret and metadata.

On a later connection, the client can offer the ticket identity in ClientHello and prove possession of the PSK. If the server accepts it, the new handshake is cryptographically tied to the original authenticated session and can omit the certificate exchange.

Earlier authenticated connection created a resumption PSKResumed protected connectionClientHelloticket identity, PSK proof, optional new key shareServerHelloPSK accepted, optional new key shareEncryptedExtensions, FinishedFinishedClientServer
6 / 6
algomaster.io

The ticket is not simply a reusable copy of an old traffic key. TLS derives new connection secrets. When resumption also performs a fresh ephemeral key exchange, the new application traffic gains forward secrecy from that exchange.

Tickets have finite lifetimes and operational consequences. A service cluster must be able to decrypt or look up tickets consistently, and ticket-protection keys must be managed securely. Losing ticket state should cause a full handshake, not an unauthenticated connection.

Zero-RTT Early Data

TLS 1.3 optionally allows a resuming client to send 0-RTT early data immediately with its ClientHello, before receiving a new ServerHello.

This saves latency, but early data has weaker security properties:

  • It does not have the forward secrecy supplied by the new handshake's ephemeral key exchange.
  • It can be replayed by an attacker across connection attempts.
  • The server may reject it, requiring application-level handling.

An operation such as “transfer ₹10,000” or “create order” must not be sent as early data merely because the request uses an idempotency-looking protocol method. The application protocol and server deployment need explicit replay-safe rules. Ordinary 1-RTT application data does not have this cross-connection replay weakness.

0-RTT is an optional optimization, not the normal meaning of TLS 1.3.

Mutual TLS

Most public HTTPS connections authenticate only the server at the TLS layer. The application later authenticates the user with a cookie, token, password, or another mechanism.

With mutual TLS (mTLS), the server sends CertificateRequest. The client responds with its own Certificate and CertificateVerify messages before its Finished message.

The client's signature proves possession of the private key corresponding to its certificate and binds that proof to the current handshake.

mTLS is useful for service identities, managed devices, and environments where both ends can securely provision private keys and certificates. It authenticates a certificate identity; the application still decides what that identity is authorized to do.

TLS 1.3 Compared with TLS 1.2

TLS 1.2 can still be deployed securely with carefully selected algorithms and configuration, but TLS 1.3 deliberately removes many legacy choices.

DimensionTLS 1.2TLS 1.3
Typical full-handshake latencyTwo round trips before client application dataOne round trip before client application data
Cipher suite meaningOften combines key exchange, authentication, encryption, and hashSelects AEAD and HKDF hash; key exchange and signatures are separate
Full-handshake forward secrecyDepends on selecting an ephemeral key exchangeRequired for public-key key exchange
Server certificate visibilityNormally plaintext during the handshakeEncrypted after ServerHello
Legacy algorithmsSupports older modes that require careful exclusionRemoves static RSA key exchange, CBC suites, RC4, and TLS-level compression
Resumption modelSession IDs or session ticketsPSK-based resumption
Early application dataNot part of standard TLS 1.2Optional 0-RTT with weaker replay properties

TLS 1.2's longer full handshake commonly follows this pattern:

The exact messages depend on the chosen cipher suite. In TLS 1.3, the server sends its key share immediately, encrypts most of its remaining flight, and removes obsolete negotiation branches.

TLS 1.0 and TLS 1.1 are deprecated, and SSL must not be negotiated. Enabling them for an old client expands the attack surface of the service rather than turning modern TLS into “backward-compatible security.”

Graceful Closure and Alerts

TLS reports protocol and security failures with alerts. Examples include an unsupported protocol version, failed handshake negotiation, a rejected certificate, or invalid protected data.

Once suitable traffic keys exist, alerts are encrypted. Some errors are intentionally reported without detailed distinctions because giving a remote peer precise cryptographic failure information can itself be dangerous.

For an orderly shutdown, an endpoint sends the close_notify alert to state that it will send no more data on the connection. This lets the recipient distinguish a complete TLS stream from a transport connection that was cut off.

Simply receiving TCP FIN does not cryptographically prove that all expected TLS-protected data arrived. Applications with meaningful end-of-message framing still need to verify that their application exchange is complete, but close_notify supplies the TLS-level closure signal.

Inspecting a Real TLS Connection

OpenSSL's client tool can expose the parameters negotiated with a server:

-connect selects the network endpoint. -servername sends SNI and is important when the address hosts multiple domains. -tls1_3 constrains this diagnostic connection to TLS 1.3, while -alpn h2 offers HTTP/2.

A successful connection reports values similar to:

The exact cipher suite, certificate, signature, and key-exchange group depend on the server and local OpenSSL build.

Interpret the fields separately:

Remove -brief to inspect more certificate and session details. Diagnostic flags should not be confused with application configuration: production software should use its platform TLS library, enable peer verification, and set the intended service identity rather than parsing s_client output.

Reading Common Failures

No shared protocol version means the client and server could not agree on an enabled TLS version.

No shared cipher or suitable key share means their allowed algorithms or groups do not intersect, or policy rejected the intersection.

Certificate verification failed means encryption may have started, but the client could not establish the intended authenticated server identity. Treating the connection as trusted anyway removes a core TLS guarantee.

The correct IP returns the wrong certificate often means the client omitted or sent the wrong SNI value and reached a default virtual host.

TLS succeeds but the application fails immediately can indicate an ALPN disagreement. The peers have a secure byte channel but do not agree on how to interpret the bytes.

A proxy reports TLS success while the backend hop is plaintext means TLS is terminating at the proxy as designed. The network protection boundary is the set of actual TLS connections, not the original URL alone.

Common Misunderstandings

TLS does not encrypt before any handshake bytes are visible. A normal TLS 1.3 ClientHello and ServerHello are observable. ECH can protect sensitive ClientHello content when supported and configured.

A certificate does not encrypt bulk application data. It normally helps authenticate the peer. Symmetric traffic keys protect records.

The server's certificate key is not the TLS 1.3 session key. Ephemeral key agreement and HKDF derive fresh connection secrets.

The padlock does not mean the application is trustworthy. It means the client established a protected connection to an authenticated endpoint under its trust policy.

TLS does not protect every hop after a proxy terminates it. Each proxy-to-service or service-to-service hop needs its own protection when the network model requires it.

A TLS record is not a TCP packet or application message. The boundaries can all differ.

The cipher suite does not describe the whole TLS 1.3 handshake. Key exchange, signature algorithms, and certificate types are negotiated separately.

Forward secrecy does not protect a currently compromised endpoint. It limits what later compromise of long-term authentication keys can reveal about erased past session secrets.

Session resumption does not reuse the old record-encryption key. The PSK participates in deriving new connection secrets.

0-RTT is not simply “faster TLS.” It deliberately trades away important security properties and is unsafe for replay-sensitive operations without an explicit application design.

Encryption is not authentication. A client that disables certificate and identity verification can create an encrypted channel to an attacker.

SSL and TLS are not interchangeable protocol versions. “SSL” survives in product names and informal speech, but SSL protocols themselves are obsolete.

Summary

TLS creates an authenticated, confidential, integrity-protected channel between two endpoints. TLS 1.3 combines ephemeral key agreement, transcript-bound authentication, HKDF, and AEAD. ClientHello and ServerHello negotiate parameters and key inputs; Certificate, CertificateVerify, and Finished authenticate the server and bind the exchange.

Separate secrets protect each direction and handshake or application phase. The record layer frames and authenticates bytes independently of TCP segments or application messages. Resumption derives a PSK to reduce setup, while optional 0-RTT has weaker replay and forward-secrecy properties.

mTLS adds client-certificate authentication but leaves authorization to the application. TLS protects only its actual endpoint-to-endpoint connection and does not hide all metadata.

TLS is an authenticated handshake that creates fresh symmetric keys, followed by a record protocol protecting the application byte stream.

Quiz

TLS/SSL Deep Dive Quiz

5 quizzes