A service can run over a 10 Gbps network and still feel slow. A transcontinental connection can have excellent throughput and still respond noticeably later than a nearby one. A system can process thousands of requests per second even though each individual request takes several hundred milliseconds.

These observations are not contradictory. They describe different dimensions of network performance:

The terms are related, so they are often used interchangeably in casual conversation. That creates vague statements such as "we need a faster network." A useful performance discussion instead asks: Is one request taking too long? Is the path unable to carry enough traffic? Or is the application achieving less than the available capacity?

Latency is a duration. Bandwidth and throughput are rates.

That distinction is the starting point, but it is not enough by itself. Every measurement also needs a boundary. "The latency is 40 ms" is incomplete unless we know whether that means one-way network delay, round-trip time, or complete API response time. Similarly, throughput might refer to link-layer bits, application payload bytes, or completed requests.

Bandwidth limits potential transfer rate. Throughput is the realized rate. Latency measures elapsed time for a particular observation.

Latency is the delay between a defined starting event and a defined ending event:

The events depend on what is being measured.

One-Way Latency

One-way latency measures the time for data to travel from sender to receiver:

Accurate one-way measurement requires the sender's and receiver's clocks to be closely synchronized. If one clock is 10 ms ahead of the other, the result is wrong by 10 ms. This is why one-way latency is harder to measure reliably across separate machines.

Round-Trip Time

Round-trip time, or RTT, measures the time from sending data to receiving a corresponding reply:

Both timestamps can come from the client's clock, so synchronized clocks are unnecessary. A ping result is normally an RTT measurement: it times an ICMP echo request and its echo reply.

RTT is not necessarily twice the one-way latency. Traffic can take different routes in each direction, and the two directions can have different levels of contention.

Request Latency

For a backend engineer, latency often means the time for a complete application operation:

This measurement can include network travel, connection setup, server processing, waiting in queues, and response transfer. It is therefore not the same as network RTT.

For example, ping can report an RTT of 20 ms while an API call takes 170 ms. The two values can both be correct. The API measurement covers substantially more work than the echo exchange.

Time to First Byte and Total Time

Even one HTTP request has more than one useful latency boundary.

Time to first byte (TTFB) measures from the start of the request until the first response byte arrives. It captures how long the client waits before the response begins.

Total request time measures until the entire response has arrived. For a large response, the gap between TTFB and total time represents the time required to transfer the body to that client.

A latency number is meaningful only when its start and end events are clear.

A service does not have one permanent latency. Each operation produces a value, and those values form a distribution.

Consider ten request latencies:

Most requests finish quickly, but two are much slower. Reporting only the average hides the user experience:

Percentiles provide more context:

Exact percentile results depend on the calculation method and sample size, so tiny samples should not be treated as precise production statistics. The important idea is that a typical latency and a tail latency answer different questions.

For an API, "p50 is 25 ms and p99 is 600 ms" is much more informative than "latency is 30 ms." It says that the common case is fast while a small but significant set of requests is slow. Interactive applications usually care about both the typical experience and the tail.

In networking, bandwidth commonly means the maximum rate at which a link or path can carry data. It is usually expressed in bits per second:

A network interface advertised as 1 Gbps has a nominal signaling rate of one billion bits per second. That number describes capacity, not the amount of application data a program is guaranteed to receive every second.

Strictly speaking, bandwidth in signal processing refers to a range of frequencies. Networking discussions also use the term for data-carrying capacity, which is the meaning used throughout this chapter.

Path Bandwidth Is Limited by the Narrowest Link

End-to-end traffic crosses several links. The path cannot sustain a rate higher than its slowest relevant link.

The 100 Mbps access link is the bottleneck in this simplified path. Replacing the server's 1 Gbps link with a 10 Gbps link would not raise this client's end-to-end path bandwidth because the narrower 100 Mbps link would remain.

Real paths are shared and can change. A nominal 100 Mbps access link may have less capacity available to one flow because other users, other applications, or provider traffic consume part of it. It is useful to distinguish:

Neither value is the same as actual throughput.

Network capacity is usually advertised in bits per second, while file sizes and application buffers are usually measured in bytes. One byte contains eight bits:

Even under ideal conditions, a 100 Mbps link cannot transfer 100 megabytes of application data each second. Its theoretical bit-to-byte conversion is 12.5 megabytes per second before accounting for protocol overhead or competing traffic.

Unit prefixes can add another source of confusion:

Network rates conventionally use decimal prefixes such as Mbps and Gbps. Some file-size tools use binary prefixes, sometimes while displaying the less precise label MB. A careful calculation states which convention it uses.

For example, the ideal transfer time for a 250 MB file over a 100 Mbps path is:

Twenty seconds is a lower bound under the simplified assumptions. Actual transfer time is longer when the usable rate is below the nominal bandwidth.

Throughput is the amount of data or work successfully delivered during an observation interval:

If an application receives a 250 MB file in 25 seconds:

If the path's nominal bandwidth is 100 Mbps, the application achieved 80% of that advertised rate:

Throughput is normally lower than nominal bandwidth. Capacity can be consumed by protocol headers, retransmitted data, control traffic, other users, or periods when the sender has nothing ready to send. The application, operating system, transport protocol, receiver, and intermediate path can each limit the achieved rate.

Throughput Depends on the Measurement Layer

The phrase "we transferred 90 Mbps" is incomplete without stating what counted toward the 90 Mbps.

A network interface may count every transmitted frame bit. A transport metric may count delivered transport data. An application may count only file contents or response bodies. These measurements observe different boundaries and can produce different values for the same transfer.

For application performance, goodput is often the most useful rate. Goodput counts only useful application payload delivered per unit of time:

A network sends 100 MB of traffic during one interval, including 7 MB of headers and 3 MB of retransmitted payload. The receiver gets 90 MB of new application data:

Goodput is not a fourth competing capacity metric. It is a deliberately application-focused way to define achieved throughput.

Application Throughput Can Count Operations

Backend systems commonly express throughput as requests per second, messages per second, transactions per second, or records per second.

These are application throughput metrics. They are not direct measurements of network bandwidth because operations can have very different sizes.

A service handling 2,000 one-kilobyte responses per second sends much less data than a service handling 200 one-megabyte responses per second:

Requests per second is meaningful only together with the workload: request sizes, response sizes, processing work, and concurrency.

Bandwidth is an upper bound under stated conditions; throughput is an observation.

Consider a highway with space for 2,000 vehicles per hour. That is similar to capacity. If only 600 vehicles pass during an hour, the observed rate is 600 vehicles per hour. The unused lanes do not create more actual traffic.

The analogy is useful as long as it is not stretched too far. Network data does not behave exactly like cars, but the distinction between capacity and realized rate holds.

Several situations can produce low throughput on a high-bandwidth path:

The result is:

Adding bandwidth helps only when bandwidth, or something directly affected by it, is the limiting resource. It does not automatically fix slow database queries, overloaded servers, or an application that sends one small request and waits before sending the next.

Latency describes an individual operation's elapsed time. Throughput describes a rate across an interval. A system can improve one without improving the other.

High Throughput with High Latency

If every request takes 200 ms and the service processes 100 requests concurrently, then in a simplified steady state it can complete roughly:

The service has substantial throughput, but each user still waits about 200 ms. Concurrency overlaps waiting; it does not make an individual request complete faster.

Batching can create a similar tradeoff. Waiting briefly to collect many items may improve total processing efficiency and throughput while increasing the latency of each item.

Low Latency with Low Throughput

A monitoring endpoint may return a tiny response in 2 ms but accept only one request at a time. Individual requests are fast when the endpoint is idle, yet the service may support little total load.

A network can likewise deliver a small packet quickly while being unable to sustain a large transfer rate. Low latency says little about how much data can be moved continuously.

Load Connects the Two

Although latency and throughput are distinct, system load often connects them.

As incoming work increases, throughput can rise while spare capacity remains. Near the system's limit, requests begin waiting for a busy link, CPU, connection pool, or downstream service. Throughput eventually approaches a ceiling, while latency can continue increasing because the queue grows.

Under light load, the arrival rate stays below service capacity, so waiting remains low and latency stays stable.

Near capacity, the arrival rate approaches service capacity. Small bursts create queues, and tail latency rises.

During overload, the arrival rate exceeds service capacity. Throughput cannot keep up, queues grow, and latency spikes.

This is why a benchmark should report both latency percentiles and throughput at the same offered load. A throughput number obtained while requests wait for seconds may not describe an acceptable service.

Increasing bandwidth reduces the time required to place a given number of bits onto the bottleneck link. That matters greatly for large transfers. It may make almost no visible difference to a small request dominated by fixed delays.

A useful simplified model for an established request-response exchange is:

The model is deliberately approximate. It separates a size-independent portion from the time needed to transfer the response.

Assume:

For the 10 KB response:

For the 100 MB response:

If achieved throughput doubles to 200 Mbps, the small response saves only 0.4 ms, while the large response saves about four seconds. More bandwidth greatly improves large-object completion time but barely changes the small request.

Latency dominates small, sequential exchanges. Transfer rate dominates large data movement.

The two effects can coexist. A remote file server might take longer to begin responding because it has a higher RTT, then finish a large download sooner because it offers greater throughput.

Consider two endpoints serving the same 50 MB object:

Using the simplified model:

Endpoint A begins the exchange sooner, but Endpoint B completes the large response much sooner.

With a 1 KB response:

For the tiny response, Endpoint A wins because fixed latency dominates. There is no universally "faster" endpoint without specifying the workload and the performance goal.

Different applications care about different combinations of the three metrics.

An interactive API usually prioritizes request latency, especially tail latency. For a small response, increasing link bandwidth alone may have little effect.

A backup or replication job moving terabytes of data cares heavily about sustained goodput. A short initial delay matters little compared with hours of transfer time.

A live audio or control application needs timely delivery. A high-capacity path is useful only if the traffic arrives within the application's deadline.

A public download service needs enough aggregate bandwidth for all clients and enough per-client throughput to complete downloads at an acceptable rate.

Capacity planning therefore starts with concrete objectives:

"Make the network faster" is not measurable. These objectives are.

Avoid confusion by labeling every result with its boundary, workload, units, and observation interval.

Measuring RTT

ping reports round-trip time for ICMP echo traffic:

A representative result might report:

This is useful evidence about echo RTT to that destination. It is not HTTP request latency, download throughput, or proof of path bandwidth. Network devices may also handle ICMP differently from application traffic.

Measuring HTTP Timing

curl can report both time to first byte and total request time:

Here, time_starttransfer and time_total are latency measurements with different end boundaries. speed_download is the average application download rate observed by curl for that transfer.

A single run is not a stable performance characterization. Caches, connection reuse, background traffic, server load, response size, and route changes can affect results. Repeat measurements under a defined workload and report the distribution.

Measuring Path Transfer Capacity

A bulk transfer tool such as iperf3 sends enough traffic to estimate achievable throughput between controlled endpoints:

The result is an achieved rate for that test, direction, protocol, duration, and number of flows. It is often used to assess a path's usable capacity, but it is still a measured throughput, not a universal guarantee for every application.

Good measurement language looks like this:

Each number answers a different question.

"A 1 Gbps connection moves one gigabyte per second." One gigabit is one-eighth of a gigabyte, and the advertised rate includes capacity that application data cannot use completely. The theoretical conversion is 125 MB/s before overhead and other limits.

"Bandwidth and throughput are synonyms." Bandwidth is capacity; throughput is the rate achieved. A 1 Gbps link carrying 200 Mbps of traffic still has 1 Gbps nominal bandwidth.

"High bandwidth means low latency." More bandwidth can shorten the transfer of a large payload, but it does not remove fixed delay. A small request can remain slow on a high-bandwidth, high-latency path.

"Low ping means the application is fast." Ping measures ICMP RTT. Application latency includes different protocol behavior and server work.

"Low average latency means nearly every request is fast." Averages can hide a slow tail. Percentiles and a sufficiently large sample reveal the distribution more clearly.

"High requests per second means high network throughput." Requests per second is an application work rate. Its corresponding data rate depends on request and response sizes.

"Throughput is one number for the entire system." Interface throughput, transport throughput, application goodput, and request throughput count different things. The measurement layer must be stated.

"A faster server link speeds up every client." End-to-end capacity is constrained by the complete path and by non-network bottlenecks. Upgrading a link that is not limiting the workload may change nothing.

Latency, bandwidth, and throughput measure different properties. Latency is elapsed time between defined events, so one-way delay, RTT, TTFB, and total request time have different boundaries. Because latency is a distribution, percentiles such as p50, p95, and p99 reveal more than an average.

Bandwidth is potential path capacity in bits per second and cannot exceed the narrowest relevant link. Throughput is the realized delivery rate, while goodput counts only useful application payload. Low latency does not imply high throughput, and additional bandwidth mainly helps when payload transfer dominates completion time.

Every result needs units, workload, measurement boundaries, and observation interval.

Replace “the network is slow” with a measurable claim about latency, capacity, throughput, or goodput.