I/O endpoints rarely produce and consume data at exactly the same rate. A process may generate a response in microseconds while the client reads it over a slow connection. A device may deliver a burst of input before the application is scheduled. Even two fast components can briefly fall out of step.

Buffering handles these short-term differences by keeping data in temporary storage. Backpressure handles the limit of that storage by slowing or stopping the component that supplies more work.

These mechanisms solve related but different problems:

A buffer buys time. It does not create processing capacity. If a producer remains faster than its consumer, every finite buffer eventually fills.

Consider an application writing many 100-byte messages to a destination. Passing each message through the entire I/O path separately can require many system calls, queue operations, wakeups, and device submissions. Combining several messages into a larger unit can reduce that fixed overhead.

Buffers also absorb bursts. Suppose a consumer normally processes 1,000 messages per second, but 2,000 messages arrive during one short interval. A queue can retain the excess while the consumer catches up.

A buffer absorbs a rate mismatch, but only a temporary one. If the producer is faster on average, no buffer size fixes it.

This decoupling improves throughput and lets each component operate in efficient units. It also has costs:

Good buffering is therefore bounded and intentional. Its capacity should reflect the burst the system needs to tolerate, not a hope that a sufficiently large queue will make overload disappear.

One logical response can wait in several places:

There are four separate buffers between one application and the other. A slow consumer fills them from the far end backwards, which is why the sending application is the last place to notice.

The application output queue holds bytes that the process has not yet handed to the kernel. The kernel send buffer holds bytes accepted by send() but not yet delivered through the rest of the path. On the receiving side, the kernel retains arrived bytes until the process reads them.

Other I/O paths have similar layers. A C standard I/O stream can buffer data in user space before calling write(). A pipe has a bounded kernel buffer between its writer and reader. Storage stacks queue requests between application calls and device execution.

The capacities at these layers do not combine into one simple “buffer size.” Each layer has different ownership and different signals:

When diagnosing queue growth, identifying the layer that owns the waiting data is more useful than saying only that “I/O is slow.”

Batching several small operations often improves throughput because fixed work is paid fewer times. If 100 records can be sent with one system call instead of 100 calls, the useful-data-to-overhead ratio improves.

Waiting to form that batch can increase latency. The first record in a batch may sit idle until enough later records arrive or a flush timer expires.

The same records reach the same destination either way. Batching trades a little delay for far fewer boundary crossings.

Queueing creates another latency cost. A request that is ready to run but has 5,000 requests ahead of it is delayed even if the consumer is healthy. Larger buffers can produce impressive short benchmarks by accepting more work quickly while increasing the time required to finish that work.

The useful question is not “Are larger buffers faster?” It is:

What burst must this buffer absorb, and how much waiting and memory can the service tolerate?

The answer depends on workload, but the trade-off is universal: buffering can raise throughput and smooth bursts, while excessive buffering increases queueing delay and resource exposure.

Let:

Ignoring rejection, the queue evolves approximately as:

If arrivals briefly exceed service, Q grows. When service later exceeds arrivals, it shrinks.

If the long-term arrival rate is below the service rate, a buffer can absorb ordinary bursts. If the long-term arrival rate exceeds the service rate, the backlog has positive net growth:

A 100 MiB queue merely delays exhaustion by roughly 50 seconds. Increasing it to 1 GiB delays the same outcome; it does not correct the rate mismatch.

At capacity, the system needs a policy. It can make the producer wait, decline new work, discard selected work, or terminate the relationship with the slow consumer. Silently allocating more memory is an unbounded queue, not a sustainable policy.

Backpressure is a signal or control action that propagates a capacity limit toward the source of work.

For a blocking pipe, the mechanism is direct: when the pipe buffer fills, a writer attempting to add more data sleeps. Once the reader frees space, the kernel wakes a writer.

For a non-blocking socket, the signal is explicit:

EAGAIN is not a reason to spin and retry immediately. It means the current execution path must stop producing writes and wait for evidence of new capacity.

The same principle appears in application queues. If a server has admitted as much work as it can safely retain, it must stop accepting more from that source or apply another overload policy.

Backpressure is successful only if it reaches something capable of reducing production. Moving items from a full kernel buffer into an unlimited application queue removes the immediate EAGAIN, but defeats the protection.

Imagine a service that reads client requests, sends jobs to a worker, and writes results to the client:

A request flows from the client, through the input, a work queue, a worker, and an output queue, and back to the client.

If the client reads results slowly, the server's output queue grows. Bounding only that queue is not enough if the server continues reading and admitting new requests whose results need more output space.

A useful propagation chain is:

The exact network protocol mechanism is outside this chapter. From the process's point of view, the important action is that it stops consuming input when it cannot safely retain the work that input creates.

Backpressure can cross several application components. A full storage queue can pause request processing; that can fill an inbound work queue; that can cause the server to stop reading or reject new requests. Every link must be bounded for the limit to propagate.

If one link uses an unlimited queue, pressure stops there and memory usage grows instead.

An event-driven server commonly controls each output queue with two thresholds:

For example:

While reading is enabled, the output queue may grow until it crosses 64 KiB. The loop then disables readable interest for that connection. It does not resume as soon as the queue drops to 63 KiB; it waits until the queue drains to 32 KiB.

The gap provides hysteresis. With a single 64 KiB threshold, a small send could move the queue just below the limit, one small read could move it above again, and the loop could repeatedly toggle its interest state.

Watermarks are not absolute memory caps by themselves. If the handler reads 16 KiB when the queue is one byte below a 64 KiB high watermark, the queue can overshoot. Bounded read sizes make that overshoot predictable. Code should also enforce a hard limit when one input unit can expand into a much larger output.

For example, a compressed request might produce a response many times its input size. Pausing the next read cannot protect against an already admitted request whose result exceeds the available budget. Admission checks and hard size limits are still necessary.

A 1 MiB output limit sounds modest until a process has 50,000 connections. If every connection can reach that allowance, the theoretical application-level exposure is about 50 GiB, before connection objects and kernel buffers are counted.

Servers therefore need limits at more than one scope:

Per-connection limits prevent one slow consumer from retaining an arbitrary amount of memory. Reaching the limit may pause that connection or close it if it cannot make progress within the service's policy.

Global limits protect the process when many connections become slow together. Once total queued output crosses a process-wide budget, the server may reduce admission, reject optional work, or choose victims according to an explicit policy.

Fairness also matters. One connection should not consume the entire global allowance merely because it arrived first or generates the largest responses. Per-connection limits reserve capacity for unrelated clients and make the global bound easier to reason about.

A practical memory estimate includes:

Not every buffer reaches its maximum simultaneously, but capacity planning should not assume that correlated slowdown is impossible. A shared downstream failure can make thousands of queues grow at once.

In a readiness-based loop, pausing a source usually means removing readable interest from the descriptor. The kernel can retain a bounded amount of incoming data while the application drains its output.

When pending output falls below the low watermark, the loop restores readable interest.

In a completion-based design, the corresponding action is to stop submitting new receive operations. Operations already in flight may still complete, so the application must reserve capacity for them. Submission depth is itself a form of buffering: every in-flight operation represents work and memory that have already been admitted.

This difference changes the API mechanics, not the resource principle. A bounded system limits both queued work and work already in flight.

Backpressure is often the preferred response to temporary slowdown, but waiting is not always safe or useful.

A server has several possible policies:

The correct choice depends on the contract of the application. Dropping a telemetry sample may be acceptable; dropping a financial operation silently is not. Pausing a producer is useful only if the producer can wait without holding another scarce resource indefinitely.

Retries require care. Immediate retry turns rejection into more load and can keep the system saturated. A rejection policy is incomplete unless callers have bounded attempts and a delay or deadline.

No overload policy makes excess demand free. It decides where the cost is paid and keeps that cost bounded.

The following Python server echoes received bytes, but bounds each connection's application output queue with high and low watermarks. It uses the standard selectors interface and non-blocking sockets.

Save the program as backpressure_echo.py and run it:

The queue grows when incoming data is accepted faster than the socket can send it back. At 64 KiB, read_paused removes EVENT_READ from that connection's interest set. EVENT_WRITE remains enabled so queued bytes can drain. At 32 KiB, the server restores read interest.

Only one READ_CHUNK is accepted per dispatch, so the high-watermark overshoot is bounded to less than one chunk. The low watermark prevents interest-state thrashing. A peer that finishes sending is not closed until its queued echo has drained.

This example makes the control mechanism visible, but a real transformation may expand input and must enforce a hard output limit as well. A production service also needs global memory accounting and a policy for connections that remain slow. Those concerns do not change the basic feedback loop.

Queue length is the most direct signal. For byte streams, measure queued bytes per connection and across the process. For request queues, measure both item count and retained bytes because a count of 100 small requests is very different from 100 large ones.

Useful observations include:

Memory use alone is a late signal. A bounded queue should expose that it is approaching capacity before the process nears an out-of-memory failure.

Throughput must be interpreted with queue size and latency. If accepted work rises while completed work stays flat and the queue grows, the system has not become faster. It has postponed the visible cost.

Buffering decouples producers and consumers, improves batching, and absorbs temporary bursts, but it consumes memory and adds queueing latency. No finite buffer can compensate for a producer that remains faster than its consumer.

Backpressure turns bounded capacity into a control signal. Blocking writers sleep, non-blocking writers observe EAGAIN, readiness loops pause input, and completion-based systems limit new submissions. High and low watermarks provide stable pause-and-resume behavior.

Reliable services bound resources at both per-connection and global scopes, preserve headroom for in-flight work, and propagate pressure far enough upstream to reduce production. A queue is safe only when its capacity and full-queue policy are explicit.