A small event-driven server can demonstrate non-blocking I/O in a few dozen lines. A production server must remain correct when clients are slow, descriptors are scarce, handlers fail, traffic arrives in bursts, and the process is asked to stop.

The event loop is only the dispatch mechanism. Dependability comes from the policies around it:

These policies turn an efficient prototype into a service that can be operated.

Production behavior is easier to reason about when a few conditions must always hold.

For each connection:

For the process:

An invariant is more useful than a best effort. “Output usually stays small” does not protect memory. “Output cannot exceed 256 KiB per connection, and total queued output cannot exceed 512 MiB” gives the server a condition it can enforce.

The limits are related. A connection cap chosen from descriptor capacity can still be unsafe for memory. A rough application-side budget is:

Kernel socket memory, worker tasks, allocator overhead, libraries, and the rest of the process require additional headroom. Limits should be tested together rather than tuned independently.

Callbacks should not infer a connection's state from which function happened to run. Store the lifecycle explicitly.

The reuse edge back to READING_REQUEST is what makes a connection long-lived. Every other state can also exit to CLOSING, which is why cleanup code has to be reachable from all of them.

A real protocol may need more states, but the important property is that each event is interpreted against the current state. Readable readiness during READING_REQUEST can extend an input buffer. A worker result during PROCESSING can create output. Writable readiness during WRITING can advance a send offset.

Unexpected events still need defined behavior. The peer can disconnect while a worker is processing. A request deadline can expire at nearly the same time as a result arrives. Shutdown can begin while output remains queued.

Treat these as ordinary transitions, not exceptional holes in the design. The connection object should record whether it is closing, which work belongs to it, and whether late results must be discarded.

A simple and robust rule is:

One event-loop thread owns a connection and is the only thread allowed to mutate its state.

If a handler needs CPU-heavy work or a blocking-only operation, it can submit an immutable input to a bounded worker. The worker returns a result through the loop's task queue. The loop checks that the connection is still valid before applying it.

The generation number is what makes the last step safe. Without it the loop cannot tell a late result for a closed connection from a valid result for a new one that reused the same slot.

Passing only an integer descriptor is unsafe. Descriptor numbers are reused. Connection A might close descriptor 42, and a newly accepted connection B might soon receive descriptor 42. A late worker result that says only “write to 42” could corrupt an unrelated session.

Use a connection object with a unique generation or request identifier. Before applying a timer, completion, or worker result, verify that its identity still matches the active connection.

Cross-thread communication also needs to wake a sleeping loop. A synchronized queue plus a pipe, eventfd, or platform equivalent lets the loop receive worker results through its normal wait path.

Accepting a connection commits resources: a descriptor, kernel socket state, an application object, buffers, timers, and future CPU time. A production server should check whether it can support that commitment.

A common flow is:

The limit check belongs at accept time. A server that accepts first and checks later has already spent the memory it was trying to protect.

The listener must be non-blocking. An accept loop normally continues until EAGAIN, subject to the readiness mode and fairness policy. Every accepted socket should become non-blocking and close-on-exec before application code can expose it to races.

On Linux, accept4() can request both flags atomically:

Portable servers use the appropriate platform calls and account for the small interval between acceptance and setting flags.

Accept errors need classification. EAGAIN means the non-blocking accept queue is currently drained. EINTR permits a retry that preserves the loop's fairness and deadline rules. A transient client-side failure such as ECONNABORTED can be counted and skipped, while resource errors require admission containment rather than an immediate tight retry.

The connection limit should remain below the process descriptor limit. The process also needs descriptors for the listener, logs, configuration, shared libraries, internal wakeup channels, and recovery operations.

Increasing RLIMIT_NOFILE without bounding accepted connections expands the failure surface. Descriptor capacity is one constraint, not an admission policy by itself.

If accept() fails with EMFILE, the process has exhausted its per-process descriptor table. The listener is still open, but the process cannot obtain a descriptor for the next connection. Repeating accept() immediately can create a CPU-consuming error loop while pending clients remain queued.

The primary protections are a connection limit, reserved headroom, and prompt cleanup. Some Unix servers add a spare descriptor as containment:

Closing the spare creates one slot, allowing the server to remove and reject one pending connection rather than spin indefinitely. Reopening it restores emergency headroom.

This technique does not repair a descriptor leak or an excessive connection limit. It makes the failure mode controlled enough for metrics, logging, and recovery to function.

System-wide exhaustion can produce ENFILE. That condition is outside the process's ability to repair; the server should avoid a tight retry loop and surface the host-level failure.

Non-blocking I/O prevents one system call from sleeping the loop. It does not stop a peer from occupying a connection forever while sending nothing or sending a partial request very slowly.

Different waits need different deadlines:

A timeout is not merely a log message. It triggers a state transition that cancels or ignores related work, releases buffers, unregisters the descriptor, and closes it.

Use a monotonic clock for elapsed-time deadlines. Wall-clock time can move forward or backward because of administrative changes or clock synchronization.

The event loop waits no longer than the nearest deadline:

For a modest connection count, a min-heap of (deadline, connection_id, generation) entries works well. Updating a deadline can leave an old heap entry behind; when that entry reaches the front, the loop compares its generation with the connection's current timer generation and discards it if stale.

This lazy invalidation avoids searching the heap to remove every obsolete timer:

If deadlines are refreshed frequently, stale heap entries can accumulate faster than they expire. The implementation must periodically rebuild the heap or otherwise bound that storage. Larger runtimes may use hierarchical timing wheels or other timer structures, but the correctness rule is unchanged: an expired timer may act only on the connection state for which it was created.

One ready client can continuously supply input. An accept queue can contain thousands of connections. A worker completion queue can receive a large batch at once. Processing an unbounded source until it is empty can delay timers and unrelated clients.

Use explicit per-turn budgets, such as:

The budget is a fairness tool, not a data-loss policy. Unfinished work remains registered or is placed on an internal runnable queue.

Readiness semantics matter. In an edge-triggered design, stopping before a descriptor reaches EAGAIN can leave it ready without producing another edge. The server must either drain it, rearm it correctly, or preserve an explicit runnable item so work continues without waiting for a new kernel notification.

Timers need a budget too. If 100,000 idle connections expire at the same instant, closing every one before handling any I/O creates a visible stall. The loop can close a bounded batch, process other events, and continue expiration on the next turn.

Network errors are normal operating conditions. Clients disconnect during reads and writes, readiness can become stale before a handler runs, and signals can interrupt waits.

A handler generally distinguishes:

Progress: a positive byte count advances an offset.

Orderly peer close: a zero-byte stream read means no more input will arrive. Pending output and protocol policy determine whether the server drains or closes.

Temporary unavailability: EAGAIN or EWOULDBLOCK means preserve state and wait for the relevant event.

Interruption: EINTR usually means retry while preserving the original deadline.

Connection failure: errors such as ECONNRESET or EPIPE normally transition the connection to cleanup.

On Unix, writing to a closed stream can also generate SIGPIPE. Servers commonly ignore that signal process-wide or use a platform mechanism such as Linux's MSG_NOSIGNAL, then handle the returned EPIPE.

Readiness systems can report error or hangup conditions together with readable or writable state. The handler should interpret all returned flags and attempt the operations needed to discover remaining data or the socket error. Assuming that one event contains exactly one condition loses valid transitions.

Error logs should be classified and rate-limited. A client reset is useful as a counter but can be too common for a full synchronous log entry on every occurrence.

Resource leaks usually come from error paths that release most, but not all, of a connection's state.

Use one idempotent close routine:

“Idempotent” means a second call has no additional effect. It is valuable because several events can independently decide that a connection should close.

Cleanup must update aggregate accounting as well as free objects. If a 128 KiB output queue is released but the global queued_output_bytes counter is not decremented, the server may believe it is permanently overloaded.

The close routine should not wait for worker tasks synchronously. Mark their results obsolete and let workers finish or honor cooperative cancellation according to the work API.

Non-blocking sockets do not make every dependency non-blocking. File access, name resolution, logging, compression, library calls, and lock acquisition can still stall the loop.

The practical rule is to measure handler wall time and inspect every operation whose latency is not tightly bounded. Move unsuitable work to a bounded worker facility or use a genuine asynchronous interface.

Offloading changes where overload appears. If the worker pool has eight threads and requests arrive faster than those threads finish, its queue fills. That queue needs a capacity, a submission policy, and a deadline.

The event loop should never block waiting for a worker-queue slot. Doing so converts worker saturation into total I/O unresponsiveness. It can pause the relevant input, reject the request, or apply another bounded policy.

Logging deserves the same treatment. A pipe to a log collector can fill, and a filesystem can stall. A bounded logging queue with a deliberate drop or fallback policy is safer than synchronous unbounded logging from the loop.

One event-loop thread executes callbacks on one CPU at a time. A production server can run several loops, commonly one per selected CPU or a small number derived from measurement.

Two topologies are common.

In an acceptor-dispatch design, one loop owns the listener and assigns each accepted connection to another loop:

Each connection has exactly one owning loop. That single ownership is what removes the need for locking around per-connection state.

This gives the application explicit placement but adds cross-thread handoff and can make the acceptor a bottleneck.

In a shared-listener design, several loop threads accept from a shared endpoint or separate listeners configured for kernel distribution. Linux SO_REUSEPORT can give each loop its own listening socket on the same address. Exact distribution and option semantics are platform-specific.

Whichever topology is used, keep a connection pinned to one loop when possible. Migrating it requires transferring readiness registration, timers, pending tasks, buffers, and ownership without racing old events.

Shared global counters can also become a source of cache contention. Per-loop counters can be aggregated periodically, while truly global limits use synchronization designed for their update rate.

More loops do not fix a blocking callback. They reduce the fraction of connections affected by one stalled loop, but every connection assigned to that loop still waits.

Immediate process exit drops active connections and abandons in-flight work. Waiting forever for every client is not operationally safe either. Graceful shutdown needs a deadline.

While draining, the server stops accepting, finishes eligible active work, and rejects new application work. The deadline edge to FORCING is what stops a shutdown from hanging on one slow connection.

When shutdown begins, the server normally:

Signal handling must wake the event loop safely. A minimal signal handler can set a flag and write to a self-pipe; Linux can also expose signals through signalfd. Complex cleanup should run in the normal loop context, not inside an asynchronous signal handler.

Shutdown accounting should distinguish accepted connections, active requests, queued worker tasks, and pending output. “No active request handlers” does not necessarily mean all response bytes have drained.

Health reporting should reflect lifecycle state. A draining instance can remain alive while reporting that it is no longer ready to receive new work.

The central loop can be expressed without tying it to one readiness API:

Several production requirements are visible here:

Real code also needs error handling around the wait itself, a wakeup channel for cross-thread tasks and signals, and a way to continue an event batch that exceeds the current fairness budget.

Request latency alone cannot explain why an event-driven server is slow. A useful operational view separates admission, queueing, handler execution, kernel waiting, and downstream delay.

At minimum, observe:

Use latency distributions rather than only averages. A 100-millisecond callback once per second can severely delay a subset of clients while the average handler remains fast.

Loop lag can be measured with a periodic monotonic timer:

High loop lag with high CPU suggests long callbacks or excessive event volume. High loop lag with low CPU can indicate a blocking wait, lock, filesystem call, or scheduler delay.

Metrics collection itself must remain bounded. Avoid unbounded label values such as connection IDs, and do not synchronously emit a log or metric for every normal byte transfer.

A load test with cooperative clients demonstrates the best case. Production testing also needs clients and dependencies that behave badly.

Useful scenarios include:

For each scenario, verify resource bounds as well as client-visible behavior. Active descriptors, queued bytes, pending timers, worker-queue depth, and memory should return toward their baseline after the test.

Shutdown testing should confirm both paths: normal drain completes before the deadline, and forced cleanup terminates within the configured bound when a peer never cooperates.

The goal is not to make every operation succeed during overload. It is to make failure predictable while preserving the server's ability to observe, reject, drain, and recover.

A production event-driven server is built around explicit ownership, lifecycle states, deadlines, and resource bounds. Accepting a connection is an admission decision, every delayed event must be validated against current connection identity, and one idempotent cleanup path must release all associated state.

Handlers and cross-thread queues require work budgets so one source cannot monopolize the loop. Descriptor exhaustion, disconnects, partial I/O, stale results, and slow dependencies are normal lifecycle inputs that need controlled responses.

Multiple loops can use several CPU cores while keeping connections pinned to one owner. Graceful shutdown stops admission, drains eligible work until a deadline, then forces cleanup. Metrics for loop lag, handler duration, queueing, descriptors, deadlines, and shutdown behavior make these controls observable.

The production objective is not to make overload and failure disappear. It is to keep resource use bounded and make the server's behavior predictable when they occur.