Long Polling

Imagine a notification system. The client wants to know whenever a new notification arrives, and users should see it soon after it lands.

The simplest approach is regular polling: ask the server every few seconds. That forces an awkward trade-off. Ask often and most requests return nothing. Ask slowly and updates feel delayed.

Long polling improves that trade-off. The client sends a normal HTTP request. If there is no new data yet, the server waits before responding. It returns when new data arrives or when a timeout fires. As soon as the client receives a response, it sends the next request.

This chapter covers how long polling works, how to choose its timeout, how to build and scale the server side, how to avoid missed events, how to handle failures, and when long polling is a good fit.

With regular polling, also called short polling, the client sends an HTTP request every few seconds and asks, "Do I have anything new?"

The server checks. If there is a notification, it returns it. If not, it returns an empty response. The client waits a few seconds and asks again.

This is easy to build, but it has two problems.

Most requests may be useless. If notifications are rare but the client polls every second, the server processes a huge number of requests that return nothing. For example, 100,000 clients polling every 5 seconds creates about 20,000 requests per second even when there are no notifications at all.

There is always some delay. If the client polls every 10 seconds, a notification may sit on the server for almost 10 seconds before the client learns about it.

Polling forces a choice between lower latency and fewer requests. Poll faster and you waste more requests. Poll slower and users wait longer. Long polling tries to improve that trade-off.

With long polling, the client still sends a normal HTTP request, such as:

The after value tells the server the last event the client has already processed.

If newer events already exist, the server responds immediately. If not, it does not respond right away. It keeps the request open and waits until either new data becomes available or a timeout is reached.

When new data arrives, the server sends the response immediately. The client processes it and then opens another long polling request with the latest event ID it has seen.

Long polling is not one permanent connection. It is a chain of normal HTTP requests, where each request may wait before returning.

A single request goes through three phases:

Compared with regular polling:

A server should not keep a long polling request open forever. Proxies, load balancers, servers, and clients all have timeout limits, so the server usually holds the request for a fixed period, such as 30 seconds. When that period ends with no data, it returns an empty response and the client immediately asks again.

Picking the value is a trade-off:

A hold timeout of 20 to 60 seconds is common, but the right value depends on the infrastructure around your server. Section 7 covers how to line it up with every layer in the path.

Start with a single application server. If 10,000 clients are long polling, the server may have close to 10,000 open requests waiting for events.

When an event arrives, the server finds the matching request, returns the response, and the client immediately reconnects. So the server must manage many waiting connections efficiently.

Older thread-per-request servers struggle with this. Each open request occupies a dedicated thread, even while it is doing nothing but waiting. With many clients, that model wastes memory and can hit thread limits.

Modern asynchronous, event-driven servers handle this much better. They use event loops that wait for network events, so a waiting request does not tie up an operating system thread. Good fits include Node.js with async handlers, Go with goroutines, async Python frameworks, Java or Kotlin reactive servers, JVM virtual threads, and Nginx/OpenResty-style servers.

Still, every open request consumes resources: memory, a socket, a file descriptor, and connection-management overhead. Long polling can be more efficient than frequent short polling, but it is not free. It replaces many empty requests with many open requests.

At a high level, the handler does this:

When one server is no longer enough, place multiple application servers behind a load balancer.

Now there is a new problem. Say a client's pending request is waiting on server 3. Another backend service generates an event for that client. How does the event reach server 3? The service that produced it has no idea which server holds the request.

This is a distributed routing problem. A common solution is shared messaging infrastructure such as Redis Pub/Sub, Kafka, or another message broker:

This separates event production from HTTP delivery. Producers do not need to know which server holds which client, and you can add application servers without changing them.

The event store and the broker serve different purposes. The event store is the source of truth that prevents missed events. The broker wakes up waiting requests quickly. Do not rely only on the broker for correctness. Pub/sub messages can be missed. If a server misses one, the client's next request must still be able to recover from storage.

Long polling has one subtle problem: the client is not continuously connected. There is always a small gap between receiving one response and sending the next request.

If an event arrives during that gap, it could be lost if the server only delivers events to currently open requests.

The fix is a cursor. Each event gets a sequence number, event ID, or timestamp. The client includes the last one it processed in every request: "Give me everything after event 43." The server checks reliable storage before waiting, so even if newer events arrived while the client was reconnecting, it can still return them.

The same cursor lets the client recover after a temporary network failure. It reconnects, sends its last event ID, and picks up where it left off.

For most systems, an increasing event ID is better than a timestamp. Timestamps can collide, move backward across machines, or create confusing edge cases.

Infrastructure timeouts matter more with long polling than with normal requests, because a long poll is supposed to sit idle.

Say the client expects the server to hold a request for 30 seconds, but the load balancer has a 20-second idle timeout. The load balancer closes the connection at 20 seconds, and the client sees a failure even though the application is working fine.

The same issue can happen with reverse proxies, API gateways, CDNs, and client HTTP libraries. Every layer in the path must use a timeout compatible with the long polling duration.

A common approach is to set the application's hold timeout slightly shorter than every infrastructure timeout. Then the server closes the request intentionally, with a clean empty response, before any intermediary cuts it off.

For example, to support a 30-second hold behind Nginx, configure the proxy to outlast the server:

The client's own request timeout should also be longer than the server's hold. If the client gives up first, it cancels healthy requests before the server has a chance to respond.

Because long polling uses standard HTTP requests, failures are relatively easy to handle. If a request fails, the client can simply retry.

But retries should be controlled. If a large cluster fails and hundreds of thousands of clients reconnect at the same moment, they can overload the system while it is trying to recover. This is a reconnect storm.

Clients should spread out reconnection attempts:

Normal empty timeout responses are not failures. After one of those, the client should reconnect immediately.

Important state should also not live only on one application server. The client's position in the event stream belongs in reliable storage, keyed by its cursor. That way a client can reconnect to any server, not just the one that failed, and resume from its last known event.

A basic client needs to handle four things: data, empty timeouts, errors with backoff, and cancellation.

The client timeout here is 35 seconds, slightly longer than a 30-second server hold. Backoff starts at 1 second, doubles on each consecutive failure up to 30 seconds, and resets after any successful response.

Long polling works well when you need near-real-time updates but want to stay within the standard HTTP request-response model. Good examples:

Compared with regular polling, it gives lower latency and avoids many unnecessary requests, without needing a fully persistent streaming connection.

Regular polling forces a choice between latency and wasted requests. Long polling improves that trade-off: the server holds each request until data arrives or a timeout fires, and the client immediately reconnects.

The hold timeout should be long enough to avoid needless reconnects and slightly shorter than every proxy, load balancer, and client timeout in the path.

Async, event-driven servers handle many waiting requests without one thread per connection, but open requests still cost memory and sockets. Across multiple servers, a message broker such as Redis or Kafka routes each event to the server holding the client's request.

Cursors close the gap between requests, so the server should always check reliable storage before waiting. When failures happen, clients retry with exponential backoff and jitter, and can resume on any server from their last event ID.

Long polling fits notifications, job status, and async results: near-real-time updates that stay inside plain HTTP.