Retrying a slow request seems like a reasonable way to recover, but many callers doing it together can make the problem worse. Suppose a bookstore's pricing service slows down during a deployment. Every API instance waits, times out, and immediately tries again. The pricing service now receives extra requests while it is already struggling. A recovery mechanism has become another source of load.

Timeouts limit waiting, retries provide another opportunity to succeed, and backoff spaces those opportunities apart. They work well only as one bounded policy.

This chapter explains how to choose timeout scopes, schedule retries with exponential backoff and jitter, respect server guidance, and stop when another attempt is unsafe or no longer useful.

The examples use a fictional bookstore over HTTPS at api.bookstore.example. Numerical settings are application choices, not HTTP defaults.

A timeout limits the duration of a particular wait. Libraries use similar option names for different boundaries, so inspect what a setting actually covers before relying on it.

These scopes overlap; their configured values are not necessarily additive. A 100 ms connection limit inside a 500 ms complete-attempt limit leaves less than 500 ms after connection establishment.

A read inactivity timeout is especially easy to misinterpret. If a server sends one byte every four seconds, a five-second inactivity limit might never fire. A complete-response deadline still needs to bound how long the caller waits for the whole document. Streaming operations need their own distinction between acceptable idle time and useful stream lifetime.

Treat a timeout as evidence that the caller stopped waiting. It is not proof that the server stopped working, rolled back a transaction, or never received the request. Local cancellation should release resources and propagate where the stack supports it, but remote completion can race with it.

Choose timeouts from the caller's remaining time and the dependency's observed behavior. A value based only on typical latency can reject healthy requests in the tail. An excessively long value holds resources and leaves little opportunity to recover.

Suppose a regional pricing lookup has a measured p99.9 of 140 ms under representative load. A 180 ms attempt limit could be an initial candidate if it fits the calling operation's budget. It is not a prediction that exactly 0.1% of future calls will time out. New connections, cold instances, traffic bursts, and changes in the dependency can alter that distribution.

Measure connection setup and pool waiting as well as server processing. Otherwise, an apparently generous timeout may fail mostly after deployments, when clients must establish many connections at once. Test reused and new connections explicitly.

A deadline is the time at which the operation's allowance ends. Establish it once using a monotonic clock for local elapsed-time calculations. Each attempt receives the smaller of its configured limit and the remaining allowance, while reserving time to return the result.

Across services, propagate remaining time through the mechanism the stack supports, and enforce local maximums. A process-local monotonic timestamp has no shared meaning on another machine. External callers also should not be able to request unlimited server work by supplying a distant deadline.

When too little time remains for useful work, stop before sending another request. Beginning an attempt with a one-millisecond allowance often spends dependency capacity without a realistic chance of helping the caller.

A retry is another attempt at the same logical operation. Its eligibility has two independent parts: repetition must be safe, and the failure must be one another attempt could reasonably resolve.

An idempotent operation preserves its intended effect when a client repeats it. HTTP advises against automatic retries of non-idempotent methods without knowledge that the operation is idempotent or evidence that the server did not apply it. A timeout alone provides neither.

For a bookstore order submission, a documented idempotency contract can provide repeat protection. Preserve its operation key, original payload, and conditions across attempts. A new key identifies a new operation. If protection is unavailable or expired, an ambiguous write needs outcome recovery rather than blind resubmission.

For timing policy, a compact classification is enough:

This table is a client policy, not a universal classification of every response. A documented application error can be more specific than its status. Unknown failures should not automatically enter an unrestricted “network error” retry branch.

HTTP also discourages automatically retrying a failed automatic retry. Multiple retries therefore require a deliberate application policy with understood semantics and limits, rather than treating HTTP as a general instruction to keep trying. The read policy this chapter develops explicitly permits up to three attempts for selected temporary failures.

The request must be reproducible too. You cannot resend an upload stream you have already consumed unless you reopen it or retain its contents. Likewise, the client cannot simply concatenate a partially consumed response with a new response unless the API supports a defined resume mechanism.

Use maximum attempts to include the initial request. Three maximum attempts means one initial attempt and at most two retries. Configuration that says only “three retries” is easy to interpret differently across layers.

Consider a book-detail read with a 1,000 ms recovery deadline, a 250 ms complete-attempt limit, and a 50 ms allowance for local completion. The deadline is a hard limit on this client's recovery effort, not a promise that a one-second response meets a faster latency target.

One possible execution is:

The entire operation finishes in 890 ms. Giving every retry a fresh 1,000 ms deadline would defeat the overall limit.

The flow below shows where time and cancellation checks belong:

Recheck after waiting because cancellation, scheduler delays, or shared retry limits can change while the task sleeps. Never sleep after the final allowed attempt: there is no next request to schedule.

Backoff is a delay before another attempt. With exponential backoff, the delay window grows by a fixed factor after each failure until it reaches a cap. This example doubles the window. Let r = 0 identify the first retry, immediately after the initial attempt fails:

The cap limits each delay window, not the number of attempts or the total operation time. A capped loop can still run forever unless it has separate stopping rules. In the three-attempt example, only retry indices 0 and 1 are reachable.

If many clients fail together and all wait exactly 100 ms, they try again together. Jitter adds random variation to spread their next attempts. Full jitter chooses each wait uniformly from zero to the current window:

The worked execution's waits of 40 ms and 140 ms fit the first two windows. Individual waits need not increase even though the windows grow. For example, a client can draw 90 ms from the first window and then 20 ms from the second.

This diagram contrasts fixed waits with independent jitter draws after a shared failure:

Jitter reduces synchronization; it does not guarantee a maximum arrival rate. Very short waits remain possible with full jitter. If the application requires minimum pacing, apply an explicit floor and describe the result as a modified policy. Independently initialized randomness matters: identical deterministic seeds across clients can reproduce the same retry schedule.

Retry-After accepts either nonnegative integer seconds or an HTTP date. A 503 can indicate expected unavailability with it, and a 429 may provide it as retry guidance. The header does not make an unsafe request safe to repeat or guarantee later success.

The bookstore receives this read:

During temporary unavailability, it responds:

The JSON error fields are bookstore conventions. All HTTP bodies in this chapter are the exact single-line JSON the examples show, without trailing newlines. Suppose the complete failure response arrives 100 ms into the 1,000 ms operation. Waiting one second already exceeds the remaining allowance, so this client stops automatic retries and returns the temporary failure. It does not shorten the server's delay to fit its local deadline.

An HTTP-date alternative could be Retry-After: Sat, 05 Sep 2026 12:00:01 GMT. Convert it to a nonnegative delay using an explicit clock-skew policy, then schedule the wait with the local monotonic clock. Do not subtract an HTTP timestamp from a monotonic timestamp. If clock accuracy is uncertain, use a documented conservative rule rather than assuming immediate eligibility.

For a valid server delay, this chapter's client treats the guidance as a minimum. It combines that delay with local backoff and adds a small positive random spread:

That extra spread is an application choice. Taking only max(server_delay, local_wait) can put every client back at the same instant when the server delay dominates. The local backoff cap never truncates valid server guidance.

For an absent or malformed header, use the bounded local policy. Negative numbers, fractional seconds, or unparseable dates are not valid delay values. Treat a past date as zero additional server delay, subject to local pacing. Extremely large valid values should cause this interactive operation to stop, not overflow an integer or become an immediate retry.

The following pseudocode combines timing and eligibility without depending on one HTTP library. Durations use milliseconds. It describes the book-detail GET policy: three attempts, a 1,000 ms overall allowance, and no hidden transport retries.

send_once must enforce its deadline across pool waiting, connection work, and complete-body reception. It produces a structured result for expected HTTP and transport outcomes. Do not convert programming errors into retryable failures. stop preserves the last result and the reason recovery ended.

The 100 ms minimum attempt allowance is a policy assumption based on measurements for this read, not a protocol requirement. The loop grants up to 250 ms when available, and never starts solely because a positive amount of time remains. The 50 ms reserve accommodates local completion; an outer deadline must still enforce the overall bound if that work stalls.

Store the response metadata the client needs for diagnostics before releasing its transport resources. Ensure cleanup occurs on terminal paths and cancellation as well. A client should not hold a connection or concurrency permit while sleeping between attempts. Cleanup itself must not become an unbounded wait.

A successful response here means a complete, usable result the client receives within the attempt deadline. Implementations must distinguish that from receiving only response headers. Cancellation and request deadlines must reach the underlying I/O operation; merely racing it against a timer can leave abandoned work running locally.

Even a small per-operation limit can multiply traffic when several layers retry. If an application allows three attempts, its SDK makes three attempts for each one, and a gateway makes three upstream attempts for every SDK attempt, the dependency can receive up to 3 × 3 × 3 = 27 requests for one logical read.

The multiplication occurs across nested loops:

This is a worst-case count under the stated nested settings. Deadlines or early success may reduce it, but configuration should not rely on that accident. Choose one owner for automatic retries on a dependency call, or explicitly coordinate the layers' total allowance. Inspect SDK, proxy, and transport behavior rather than assuming an application loop is the only source of repetition.

A retry budget limits extra attempts across many operations. For example, one client process might keep a token bucket with capacity 20, initially containing 20 tokens, and add one token for every ten initial requests. Each retry consumes one token. With N initial requests, the process can admit at most 20 + floor(N / 10) retries from that initial state; bucket overflow can make the actual number smaller.

This example policy allows a short recovery burst and approximately 10% extra traffic over sustained initial traffic. The client consumes a token immediately before sending a retry, as in the loop. New attempts without tokens stop automatic recovery even if their individual attempt count allows more.

The budget's scope matters. One hundred freshly started processes with 20 tokens each can collectively admit 2,000 retries before replenishment. A per-process bucket is not a fleet-wide guarantee. Size local budgets against expected fleet size, or coordinate a broader budget where needed.

Retry budgets supplement admission and concurrency limits. They do not cap initial traffic, and backoff does not create capacity during a sustained outage. A circuit breaker, which temporarily stops calls to an unhealthy dependency, may be useful, but it needs its own recovery policy rather than replacing these basic limits.

When a retry succeeds, the caller may receive an ordinary response:

This can be the final response in the 890 ms execution. The retry may observe newer book data than the initial attempt would have returned. Successful repetition of a read does not promise an unchanged snapshot.

When recovery stops, distinguish a known rejection, exhausted read attempts, and an unresolved write outcome. “No more retries” describes client behavior. It does not establish that a timed-out write failed without effects. Keep the last useful status or transport category alongside the stop reason.

Verify policy decisions with a controllable clock, deterministic random draws, and a fake transport before exercising real networking. Then inject representative failures into integration tests:

Track logical operations and attempts separately. Record initial-attempt success, final success, attempts per operation, backoff time, cancellations, and stop reasons. Observe dependency load and latency alongside recovery rates. A rising final success rate can hide growing attempt failures and unacceptable caller waits.

Use route templates and bounded failure categories in metrics, and keep credentials and sensitive bodies out of diagnostic records. During an outage, verify that attempts stop at their configured limits and that queued or abandoned work does not prevent recovery after the dependency becomes healthy.

Bound both individual waits and the complete logical operation. Reuse one deadline, preserve request intent, and retry only when repetition is safe and the failure is a useful candidate for recovery.

Exponential backoff grows the delay window; jitter spreads attempts within it. Respect valid server timing guidance without exceeding the caller's deadline, and stop promptly on cancellation or exhaustion.

Control retries across layers and across traffic, not only within one loop. Verify complete-response timing, cleanup, and outcome reporting so that recovery improves service without overwhelming dependencies or concealing uncertain writes.