A server receives 8 KB on a socket registered with edge-triggered epoll. Its handler reads only 1 KB and returns to epoll_wait().

Seven kilobytes are still waiting, yet the server may sleep indefinitely. No new transition is required to occur because the socket never stopped being readable.

The bug comes from mixing up two questions:

The first separates readiness from completion. The second separates level-triggered from edge-triggered readiness.

These distinctions determine whether an application should attempt an operation, whether an operation has already finished, and how much work a handler must perform before waiting again.

A readiness interface watches an endpoint's current state.

For a socket, it can report conditions such as:

The application reacts by issuing the actual operation:

The kernel reports that the socket is readable, the application calls recv(), and recv() returns bytes, 0, EAGAIN, or an error.

The readiness event does not contain the requested application data. It also does not reserve the condition for the receiving thread. Another thread can consume the data first, or the endpoint can change before the operation runs.

This is why readiness-driven descriptors are normally non-blocking. The notification makes progress likely; non-blocking mode prevents a stale observation from parking the thread.

select(), poll(), Linux epoll, and BSD/macOS kqueue can all expose readiness.

A completion interface starts from a submitted operation rather than a watched condition.

The application provides an operation description:

If submission succeeds, that specific read remains in flight. A later completion reports its result:

The application does not receive “the descriptor is readable” and then issue request 42. Request 42 was already issued.

A completion normally carries or identifies:

The result can still be partial. Completion means that one submitted operation reached a terminal result, not that an entire application message or file transfer is finished.

The same logical read looks different in the two models:

The readiness model tells you when to act. The completion model tells you what already happened.

The practical differences are:

Readiness is often a natural fit for many mostly idle stream sockets because no read buffer has to be committed to every idle connection. Completion can be a natural fit when many concrete operations and buffers are already known, such as independent file reads.

Real systems can use both. A runtime might monitor sockets for readiness while using completion-based operations for file or device I/O.

Readiness and completion should not be reduced to different spellings of blocking and non-blocking.

A non-blocking read() is synchronous: it performs the current attempt and returns bytes, 0, an error, or EAGAIN. A readiness API helps the application decide when that synchronous attempt is worth making.

A completion operation is asynchronous from the application's perspective: submission returns while the operation remains in flight, and its result arrives separately.

Even then, internal implementation details can vary. A library may provide completion-style application semantics by running blocking calls on helper threads. The notification model describes what the application observes, not necessarily every mechanism underneath.

Level-triggered notification reports a condition while that condition remains true.

Suppose a socket changes from empty to containing three bytes:

The socket goes from empty to holding A, B, and C, so readiness goes from false to true and the kernel reports it readable.

The application reads one byte:

If the application waits again, a level-triggered interface can report the socket as readable again because bytes B and C remain.

The behavior resembles a condition that stays asserted:

Linux epoll uses level-triggered behavior by default. select() and poll() also effectively report current readiness levels on each call.

Level-triggering is forgiving. A handler can read one chunk, return to the wait loop, and receive another notification if the condition still exists.

That does not make ignoring events harmless. If an application never consumes a readable condition, the wait can return immediately again and again, creating a busy loop.

Edge-triggered notification focuses on a relevant state change, such as the transition from not readable to readable.

Using the same three bytes:

The socket goes from empty to holding A, B, and C, and the kernel reports that transition.

Now the application reads only A:

The socket did not return to the not-readable state. An edge-triggered API is not required to report another event merely because B and C remain unread.

The application must consume the current readiness condition before it relies on a future transition:

On Linux, EPOLLET requests edge-triggered behavior for an epoll registration.

An edge-triggered read handler should normally continue until the non-blocking operation returns EAGAIN or EWOULDBLOCK:

A short positive read is not a drain boundary. If recv() returns 200 bytes into a 4096-byte buffer, the handler should not assume the socket is empty. It continues until EAGAIN, EOF, or a real error.

EAGAIN does not promise that the socket will remain empty after the syscall returns. New data can arrive immediately. It means the handler consumed everything that was available at the instant of that attempt.

Draining applies to more than data reads.

Accept until EAGAIN

One listener notification can represent several queued connections. An edge-triggered accept handler loops:

Accepting only one connection can leave others in the accept queue without a new edge to wake the loop.

Write until complete or EAGAIN

A write-ready notification means some local output capacity exists. The handler writes from its pending buffer until:

If output remains after EAGAIN, the application preserves the buffer and offset and continues watching for writability. Once all output is accepted, it removes writable interest.

Watching EPOLLOUT continuously is a common busy-loop bug because healthy sockets are writable much of the time.

An edge-triggered handler cannot know in advance which call will consume the last available byte.

If the descriptor is blocking, the loop can behave like this:

Reading the available bytes and then calling read() once more, with no data currently available, blocks the entire event-processing thread.

With O_NONBLOCK, that final call returns EAGAIN, giving the application an explicit stopping condition.

Non-blocking mode is useful with level-triggering as well because readiness can become stale. With edge-triggering, it is fundamental to the drain-until-EAGAIN protocol.

An application must not interpret one readiness event as one packet, one message, one byte, or one peer write.

Several changes can occur before the application calls the wait function. The kernel can coalesce them into one report because the important fact is that the condition is ready:

Notifications do not correspond to sends. Code that assumes one notification means one message will mis-parse the stream.

Conversely, an application can receive more than one notification while working through changing state. Edge-triggering does not promise one perfectly preserved event record for every low-level transition.

The correct logic uses I/O return values as the source of truth. Notifications tell the application where to look; read(), write(), and accept() reveal the current result.

A readable notification can mean that a stream has reached EOF. The handler must call read() or recv() and interpret 0.

Error and hangup flags can arrive with readable data. Closing immediately on EPOLLHUP can discard bytes already queued for the application. A robust handler processes readable input, observes EOF or the relevant error, and then tears down the connection.

Level-triggered code that leaves EOF or an error unhandled can be awakened repeatedly because the condition remains reportable. Edge-triggered code can instead lose the only useful notification if it ignores the terminal state.

In both modes, readiness is broader than “payload bytes exist.”

Linux epoll also provides EPOLLONESHOT, but it solves a different problem.

A one-shot registration must be rearmed with EPOLL_CTL_MOD before more events are delivered. It is useful when an application wants one worker at a time to own a connection.

The flags can be used independently or together. Forgetting to rearm a one-shot descriptor causes a stall even if the handler drained the socket correctly.

A completion-driven handler does not drain an unspecified readiness condition. It consumes terminal results for operations that were explicitly submitted.

Suppose the application submitted:

The completion stream might report requests 12, 10, and 11 in that order. For each one, the application:

There is no edge-versus-level choice for an already submitted operation's terminal result. A completion is tied to that operation. The API must retain or make the result available until the application consumes it according to the interface's rules.

Completion APIs still require capacity management. Every in-flight operation consumes request metadata, buffer memory, and kernel or device queue space. Submitting unlimited work can increase latency or exhaust resources even though no thread blocks for each request.

The following Linux program registers one end of a non-blocking socketpair() with epoll. It sends three bytes, waits for readability, and deliberately reads only one byte.

It runs once with default level-triggering and once with EPOLLET:

Compile and run it on Linux:

Expected output:

In both runs, bytes B and C remain after the one-byte read.

The level-triggered second wait reports readiness because unread data still exists. The edge-triggered second wait times out because the program did not drain the condition and no new not-readable-to-readable transition was needed.

The demonstration then drains the socket before cleanup. Replacing the one-byte read with a loop that continues until EAGAIN produces the correct edge-triggered handler shape.

Level-triggered readiness is usually the simplest choice. Repeated reporting makes handlers tolerant of bounded reads and incremental processing. The trade-off is that an unhandled condition can wake the loop continuously.

Edge-triggered readiness can reduce repeated notifications and fit handlers that already drain non-blocking endpoints completely. It demands stricter code: every relevant operation must continue to EAGAIN, EOF, completion of pending output, or a terminal error.

Completion models make operation state explicit at submission time. They avoid the readiness-then-operation sequence, but require a buffer and request object for every in-flight operation and careful matching of out-of-order results.

The best fit depends on the endpoint and workload. Large sets of mostly idle stream connections often fit readiness well. Known file operations and devices with command queues can fit completion well. A single application can combine the models rather than forcing every kind of I/O through one abstraction.

Readiness reports that an operation may make progress; the application still calls read(), write(), or accept(). Completion reports the terminal result of an operation that the application already submitted.

Level-triggered readiness remains reportable while a condition is true. Edge-triggered readiness emphasizes state transitions and does not guarantee another notification merely because unread data remains.

Correct edge-triggered handlers use non-blocking descriptors and drain reads, writes, and accepts until EAGAIN, EOF, completion of pending work, or a real error. Short reads and event counts do not establish that an endpoint is drained.

Completion-driven code instead tracks explicit in-flight requests, buffers, byte counts, errors, and out-of-order results. It must preserve request resources until each terminal completion is consumed.

The central mental model is:

Readiness says “try the operation”; completion says “the operation finished.” Level-triggering reports a true condition, while edge-triggering reports the transition that made it true.