Starting a lengthy task should not require keeping a dashboard open until it finishes. If a partner starts a large catalog export and returns ten minutes later, it needs more than a message saying the service accepted the request. It needs to find the same work, understand its progress, retrieve the completed file, or stop the work if the partner no longer needs the file.

A long-running operation gives that work a durable identity and a lifecycle that clients can observe.

This chapter explains how to design the operation resource, state transitions, progress, results, cancellation, and retention using a fictional bookstore API over HTTPS.

A long-running operation represents work that completes independently of the request that started it. There is no universal duration threshold. Choose this model when keeping the request open for the expected duration is impractical for clients or infrastructure, or when clients need to check the work after a failure.

The operation resource is a record of execution. It is separate from the business resource that the service creates or changes. In the bookstore, /operations/op_801 tracks generating a catalog file, while /catalog-exports/export_801/file serves the successful result.

Keeping those identities separate lets the API explain a failure even when the service produced no file. It also lets the operation expire without implying that every kind of business resource that an operation creates must disappear with it.

The diagram below shows this relationship:

Every accepted export remains observable through its operation, including exports that produce no result. This example publishes a file only after generation completes successfully. Temporary output stays private; the service cleans it up separately.

For an operation that provisions a database or updates an existing resource, the business resource might be visible while work is active. Its representation should then explain whether it is usable. Do not make callers infer readiness merely from the resource's existence.

The paths, field names, lifecycle, and time limits in this chapter are choices for the fictional API. HTTP does not define a universal operation schema.

The partner submits an export request. This endpoint accepts csv or json, takes a consistent catalog snapshot when execution starts, and requires an idempotency key. All HTTP JSON bodies in these examples are compact UTF-8 without a trailing newline; tokens are nonfunctional placeholders.

Before acceptance, the service checks input, export permission, and admission capacity. It then durably records the operation and the intent to execute it:

202 Accepted means the server has accepted the request for processing but has not completed that processing. Durable recording is an additional service promise, not a guarantee that status code supplies. This API uses Location to identify the status resource; the JSON link provides the same address.

Returning 201 Created would be a possible design for an endpoint whose declared purpose is creating a job resource. That response would establish job creation, not completion of the job's business work. Here, the endpoint requests file generation, so 202 communicates the intended boundary directly.

If the client never receives the response, resubmitting equivalent validated input with the same key returns the existing operation. The service keeps a separate mapping for each partner and endpoint, for as long as the operation is active and for seven days after it finishes. Different input with that key returns 409 with idempotency_key_reused. A replay never changes the accepted input or starts a second execution merely because the original operation has finished.

Known rejections should not create operations. For example, changing the request body to {"format":"xml"} produces:

An authenticated partner without export permission receives 403 with export_not_allowed. The service also checks admission limits before dispatching work. The service must not return a no-work rejection after it has already handed the request to a worker.

An operation's state is part of the public contract. Keep it small enough for clients to handle, and distinguish active work from terminal outcomes. A terminal state means execution has ended and will not resume under that operation ID.

The bookstore uses these states:

The allowed transitions are:

A pending operation can fail if the partner loses permission or its execution deadline passes before it starts. A cancelling operation can succeed if the service had already committed publication when cancellation arrived. State names must describe the established outcome, even when it differs from the caller's preference.

Enforce transitions atomically. Two workers must not independently record succeeded and failed for the same operation. Once the service commits a terminal outcome, late progress writes and stale workers cannot overwrite it. A new deliberate attempt after failure gets a new operation ID.

Internal worker retries remain within the same operation. Exposing each retry as a new operation makes clients responsible for tracking implementation details and can make one business request look like several independent requests.

The representation also has field invariants. started_at is present once execution begins. completed_at is present only in terminal states. result_url appears only for success, and error appears only for failure. The API represents cancellation with cancelled, rather than a fabricated success result or an unexplained error.

These rules let clients determine what happened without guessing from missing fields. Clients should use status to determine the outcome. Timestamps add context; they do not replace status.

The partner checks the returned monitor:

A response during generation might be:

200 means the API successfully returned the operation. It does not mean generation is complete. This endpoint keeps the same representation shape across active and terminal states, so the client always inspects status.

Progress should measure something real. Here, the worker captures a snapshot and establishes its record count before reporting total_records. records_written counts records the worker has saved in output it can recover, rather than every attempted write across retries. The count does not decrease when a worker restarts.

Before the worker knows the total, omit total_records and expose a phase such as preparing. Do not invent a denominator to produce a percentage. A known count of 42,000 out of 100,000 describes record writing, not 42 percent of total elapsed work: compression, upload, and publication may remain afterward.

An estimated completion time is a forecast, not a deadline. If the service cannot estimate it usefully, a phase and last progress update are more honest. Avoid a progress indicator stuck at 99 percent because the estimate omitted the final phase.

updated_at records the latest public state or progress change; it is not a worker heartbeat. A long upload might legitimately leave it unchanged for a while. Worker liveness belongs in service monitoring, and a missing heartbeat alone should not make the API declare a business failure whose effects are still uncertain.

The custom poll_after_seconds field recommends when to check again. Clients add jitter, observe throttling guidance, and stop automatic polling when their own waiting budget ends. They keep the operation ID for later retrieval.

The example uses Cache-Control: no-store and authoritative status reads. That cache directive does not fix stale database replicas. If status reads can return stale data, provide a way to recognize older responses, such as a revision number that only increases. Clients can then avoid replacing newer displayed state with an older response.

The export succeeds only after its file is complete and the client can retrieve its published result. The next status request returns:

The client downloads the file through a separate authorized GET to result_url. The service could include small results in an operation response, but it should not resend large files every time someone checks status. A result link also allows file transfer to have its own response type and retry behavior.

Keep the monitor available after success. Automatically replacing its JSON response with file contents would force clients to handle unrelated representations at the same URL. A separately documented redirect design is possible, but this API consistently returns operation metadata.

Now consider a separate export, op_802, that exhausts its internal retries before publishing a file. Its status retrieval returns:

The error describes execution. Returning an outer 500 simply because this retained operation failed would confuse a known outcome with failure to read the monitor.

A temporary failure of the monitor itself leaves the operation outcome unknown to the client. Likewise, a temporary download failure after success does not undo file generation. Retry the failed observation or retrieval before considering a new export.

For this export contract, failed means the service published no file. Other operations may have partial effects. An import that created 80 records before failing needs retained item outcomes and an explicit statement that those records remain. A final status alone does not explain which changes the service saved.

Errors should explain what the caller can act on without exposing worker stack traces, credentials, or private storage paths. Whether a new submission is safe depends on the business effects and retry contract, not just an error field saying that a dependency was unavailable.

The animation below compares waiting for an export with tracking it through an operation ID.

Cancellation asks the service to stop work. It is an action on the operation, with its own authorization and outcome. This API exposes POST /operations/{id}/cancel and does not offer deletion of active operation records.

For an active export, the partner sends:

The service records the stop request and responds:

This acknowledgment is not proof that execution stopped. Workers may be finishing an in-flight action, and the service may need to establish whether result publication already committed.

The service resolves the race by deciding whether publication can commit:

The service coordinates publication and cancellation so only one outcome wins. An implementation could use a publication record that accepts a write only if cancellation has not already won. That record also determines whether the private file becomes accessible. Cancellation cannot revoke a publication that already committed merely by overwriting the operation's status.

The endpoint's repeat behavior is explicit:

Those are application rules, including the repeat behavior of POST. Retrying a lost cancellation acknowledgment is safe under this contract. A 409 after a retry can mean completion won the race; the client retrieves the operation to learn which outcome the service recorded.

Cancellation is also distinct from rollback. Stopping an import need not delete rows the import already created. Removing a completed result is another operation with its own permissions. Deleting an operation record, if an API supports it, should not silently double as cancellation or result deletion.

Three different clocks affect the caller:

The bookstore sets an execution deadline 30 minutes after acceptance, including queue time. It checks the deadline before starting execution and at the guarded publication boundary. Reaching the deadline prevents a new publication, but the terminal status may appear later while the service stops work already in progress or checks which effects committed.

If the service published nothing and the deadline wins, the operation ends as failed with execution_deadline_exceeded. If publication committed before the deadline, recovery records success even if the worker acknowledgment arrived afterward. A timeout must not manufacture a no-result claim when a result already exists.

Workers need durable progress and exclusive authority to finalize results. A common implementation uses a lease: a time-limited claim to the work. If the lease expires, another worker can recover the operation. Because the old worker may still be running, storage must reject writes from superseded claims; the clock expiring alone does not prevent duplicate publication.

A recovery process looks for abandoned work, checks durable publication state, and either resumes safely or records an established failure. Queue redelivery alone is insufficient if the operation can become stranded before a message reaches the queue.

Keep the public operation active while resolving uncertain effects. Record diagnostic details internally and alert operators when reconciliation exceeds its expected duration. Clients need an honest pending outcome more than a quick terminal label that later changes.

Operation IDs must not act as access credentials. The bookstore binds operations to the submitting partner and checks permissions separately for reading status, cancelling work, and downloading results. A token that allows its holder to view exports might still lack cancellation permission.

For example, an authorized status reader without cancellation permission receives:

For an operation belonging to another partner, this API returns 404 consistently to conceal its existence. The cancellation route follows the same concealment policy as the status route. Limit workers to data the submitting partner may access, and have the service recheck export permission before execution begins.

The example retains terminal metadata and files for seven days after completion. expires_at describes operation retention; result_expires_at describes the successful file's retention. They happen to match here, but separate fields allow clients to understand APIs with different retention periods. The terminal-record cleanup process does not remove active operations.

After expiry, this service retains an ownership-aware expiry marker for an additional seven days. During that period, authorized reads return 410 Gone with operation_expired; unauthorized reads still return 404. Once the service removes the marker, reads return 404 for everyone. The marker supports expiry reporting only and does not extend the submission key's deduplication window.

These expiry errors also use Cache-Control: no-store. An expired operation is not evidence that execution failed or never happened. Clients preserve results they need before expiry and must not blindly replay an old submission after its duplicate-protection window closes.

Verify the boundaries where different participants can disagree. Lose the acceptance response and confirm that a replay returns the same operation. Restart a worker after it stores output but before it reports success, and confirm that recovery publishes at most one result. Send a stale worker update after completion and verify that terminal state remains unchanged.

Exercise cancellation before execution, during output writing, and immediately around publication. The outcome must match the published result: cancelled cannot coexist with a newly accessible export file. Repeating cancellation should preserve the documented response rules rather than creating extra work.

Advance the execution deadline while work waits in the queue and while publication is in flight. Expire retained records and verify ownership concealment, expiry reporting, and file access independently. During progress checks, confirm that durable counts survive restarts and that a stalled heartbeat does not become an invented execution error.

Operational measurements should include queue wait, execution duration, time spent cancelling, recovery age, and completion outcomes. The initial acceptance response measures only admission. A service can return every submission quickly while leaving the actual operations stranded.

A long-running operation needs a stable identity, explicit state transitions, trustworthy progress, and a retained result or error. Keep execution outcomes separate from status retrieval, and make terminal states describe what actually committed.

Cancellation, execution deadlines, and retention solve different problems. Define their races and access rules, preserve enough state to recover worker failures, and let clients reliably check progress again after they stop waiting.