When you first build an orders API, a single service behind a public hostname may be all you need. As the system grows, separate teams may add payments, inventory, and shipping services. Clients still need a stable interface, but authentication checks, traffic limits, and routing rules now span several deployments.

An API gateway provides a shared entry point for that traffic. It can make common policies consistent, but it also introduces a dependency that can affect every API behind it.

This chapter explains where gateways fit, which responsibilities belong there, and how to preserve API behavior when requests cross this extra boundary.

An API gateway accepts API requests, applies configured policies, and forwards accepted requests to an appropriate backend. Gateway documentation often calls that backend the upstream service. From the gateway's perspective, the client connection is downstream.

Consider a commerce API exposed at api.shop.example. Clients use public paths such as /v1/orders and /v1/payments. The gateway maps those paths to internal services, so moving a service does not require clients to discover a new address.

The diagram separates that public interface from the services that implement it.

Clients depend on the public contract. Services own its business behavior. The gateway connects the two, but a stable hostname alone cannot protect clients when response formats or operation behavior change incompatibly.

Related Infrastructure

These components have overlapping capabilities. Their main purpose is more useful than a rigid product classification.

A gateway commonly acts as a reverse proxy and performs load balancing. It may sit behind another load balancer or a content delivery network. Deploying all these components is not a prerequisite for a production API.

For a small API with one service and modest policy needs, an existing reverse proxy plus application middleware may be sufficient. Introduce a gateway when shared policy or routing requirements justify another component to configure and operate.

Good gateway policies depend on information that the gateway can evaluate reliably: the requested operation, validated credentials, request size, and current traffic levels.

Business decisions usually need service-owned data. The orders service knows whether an order belongs to the caller's tenant and whether the caller can still cancel it. Moving those decisions into gateway scripts duplicates business rules and makes deployments harder to coordinate.

Schema validation at the gateway can reject malformed requests early. It should not become the only validation layer. Services may have other callers, and gateway schemas can lag behind a service deployment. A strict gateway validator can accidentally block a newly supported optional field.

Avoid turning the gateway into an order-processing engine that reserves inventory, charges a card, and sends notifications. Those steps require business state and failure recovery. Keep them in an application service with clear ownership.

Route matching should be explicit about hostnames, methods, and paths. A broad /v1/ rule can accidentally expose an administrative endpoint if it forwards every matching path to a service.

For the commerce API, the intended routing rules might be:

These are example policy choices, not a gateway configuration format. The braces represent a path parameter; each gateway has its own matching syntax and precedence rules.

Reject unknown public routes using documented behavior. Define how the gateway handles unsupported methods. Test overlapping routes, trailing slashes, and encoded path characters so the gateway and backend interpret the same request consistently. A gateway that authorizes one interpretation of a path and forwards another can create an access-control gap.

Request and Response Preservation

For ordinary forwarding, preserve the meaning of the method, query parameters, body, and relevant end-to-end headers. Gateway products have different defaults, so verify this behavior instead of assuming transparency.

For example, stripping If-Match can remove a client's concurrency protection. Dropping a service-supported idempotency key can make a retry create duplicate work. Rewriting Location incorrectly can direct clients to an internal hostname. Returning a cached personalized response to another caller can expose private data.

Some headers belong to a particular connection and cannot simply pass unchanged across proxy hops. Let the HTTP implementation handle that distinction rather than implementing a blanket “forward every header” rule.

Keep transformations narrow and documented. If a gateway changes response bodies, it must also account for affected metadata such as representation validators and content lengths. The more it changes, the more of the public contract it owns.

Suppose a client calls GET /v1/orders/ord_784 with an access token. The gateway checks the token and the route's required scope. The orders service still needs to determine whether this caller may access that specific order.

This flow assumes that the gateway forwards an access token that the orders service’s policy allows it to accept. It shows the two distinct authorization decisions.

Passing the gateway check means the caller may attempt the operation. It does not establish ownership of the requested resource.

For signed tokens, validate the signature with an allowed algorithm and trusted keys, the expected issuer, the intended audience, and applicable time constraints. A valid signature alone is insufficient. Do not forward a token intended only for the gateway and configure services to ignore its audience restriction.

Another design lets the gateway pass a protected internal identity assertion. That requires an explicit agreement about who can issue the assertion, who can receive it, its lifetime, and how services verify it. Mutual TLS can authenticate the gateway connection, but it does not by itself describe the end user's permissions.

Untrusted Headers and Direct Access

Never trust a client-supplied X-User-Id or X-Tenant-Id as proof of identity. If such custom headers carry internal identity, remove incoming values and construct replacements from verified identity information. Services should accept them only over an authenticated path from an authorized gateway. These names are application conventions, not standard authentication mechanisms.

Forwarded client-address headers need a separate trust policy. If a content delivery network sits before the gateway, configure which proxies the gateway trusts and how it interprets the address chain. Otherwise, a caller may spoof an address the gateway uses for logging or traffic limits.

Restrict incoming access to backend services so an attacker cannot bypass gateway checks by calling a public service address directly. Any intentional alternate entry point needs equivalent protections. Retain resource authorization in the service regardless of the network layout.

The simulation below shows how a gateway routes requests and applies API-key checks and rate limits.

Assume requests use HTTPS, the example token is valid for the orders API, and the caller has access to ord_784. The following HTTP/1.1 messages show a successful lookup. Token values are placeholders; the examples omit body lengths for readability.

The orders service produces the representation, which the gateway forwards:

This authenticated order API disables shared response caching. The gateway must honor that policy rather than enabling caching solely because the method is GET.

Authentication and Authorization Failures

The gateway can reject an expired bearer token before the service receives the request:

For a valid token missing orders:read, the gateway can return 403 Forbidden with WWW-Authenticate: Bearer error="insufficient_scope", scope="orders:read". A request without credentials receives a bearer challenge without claiming that a supplied token was invalid.

A caller with the right scope may still lack access to ord_784. The service rejects that request using the API's documented resource-access policy. Some APIs return 404 to conceal resource existence; others return 403. The gateway should preserve that intentional behavior.

Validation Failures

For an order-creation request whose JSON is malformed, a gateway configured to parse JSON can reject it with 400 Bad Request. The service checks whether a syntactically valid quantity exceeds a business limit.

The client-facing error format should remain consistent regardless of which component rejects the request. Configure gateway-generated errors deliberately; a default HTML proxy error is a poor fit for an API that otherwise returns JSON problem details. Preserve useful service error bodies instead of replacing every failure with a generic gateway message.

A gateway adds processing time and another place where a request can stop. Allocate its timeout as part of the full request budget, including policy checks, connection establishment, upstream work, and response transmission.

For an illustrative lookup, a client may wait two seconds while the gateway allows 1.5 seconds for upstream work. The remaining time leaves room for other processing and network delays. These values need measurement under realistic load. Check when the chosen product starts each timer: connection, overall request, per-attempt, and idle timeouts cover different intervals.

Limit retries to operations and failure conditions where repeating the request is acceptable. A proxy must not automatically retry non-idempotent requests. For this design, the gateway disables automatic retries on order and payment creation. Clients can use an explicitly designed service idempotency mechanism when retrying those operations.

Retries also consume capacity. If a client makes three attempts and the gateway makes two upstream attempts per client request, one logical operation can produce six upstream attempts. Assign retry ownership and cap total work within the available time budget.

Gateway Error Semantics

A 502 Bad Gateway describes an invalid upstream response. A 503 Service Unavailable describes temporary inability to serve the request, such as overload. A 504 Gateway Timeout means the required upstream response did not arrive in time. Map connection failures according to the implementation's documented behavior rather than treating every failure as a timeout.

A timeout does not prove that a write failed. The sequence below shows the orders service committing an order even though the client receives a gateway timeout.

The gateway cannot infer transaction state from a missing response. Preserve the service's idempotency mechanism and explain uncertain outcomes to clients. Canceling upstream work may reduce wasted processing, but cancellation does not roll back an already committed operation.

Overload and Streaming

Rate limits control arrivals; concurrency limits bound work already in progress. Use both where slow upstream calls can exhaust gateway connections or memory. Keep queues bounded so overload does not become an expanding backlog of requests that will time out anyway.

A circuit breaker temporarily stops calls under configured failure or capacity conditions. It can reduce pressure on an unhealthy upstream, but it needs limits on recovery checks and settings that suit each route.

Streaming routes need separate policies. Buffering an entire response can defeat incremental delivery, and a short whole-response timeout can terminate a healthy stream. Verify idle limits, buffering, connection draining, and protocol support. Once the gateway has sent response headers, it cannot replace the response with a new JSON error status; clients must handle stream interruption through the streaming contract.

A shared entry point should not mean a single running instance. Spread gateway replicas across appropriate failure domains and verify that remaining capacity can carry traffic during a failure or rollout. Test both requests per second and long-lived connections.

Local rate-limit counters are not automatically a global quota. If each replica permits the full limit independently, adding replicas can increase aggregate allowance. A shared limiter adds coordination and its own failure modes. Choose the required precision and availability deliberately.

Define what happens when policy dependencies fail. An authentication dependency outage should not silently grant access. A low-risk route might tolerate a bounded local fallback if a shared traffic limiter is unavailable. Those choices should be explicit per policy, with operational signals when fallback is active.

Configuration Changes

The data plane is the set of gateway instances handling requests. The control plane manages and distributes their configuration. Where the product supports it, keep the data plane serving a validated last-known-good configuration during a control-plane outage. Account separately for expired certificates, stale authorization data, and unavailable credential keys.

Version routes and policies together with their owners. Validate changes before rollout, then introduce them to a small portion of traffic and compare behavior before expanding. Monitor which configuration revision each instance has accepted. Multiple replicas do not protect against a bad configuration that every replica uses.

Useful checks exercise the public entry point: an accepted lookup, an expired token, insufficient scope, forbidden order access, malformed JSON, overlapping routes, an upstream timeout, and a dropped write response. Confirm that rollback restores the prior behavior and that deployment draining respects existing connections.

Record enough information to distinguish a gateway rejection from a service response: matched route, status, failure origin, upstream duration, attempt count, and configuration revision. Use bounded route names in metrics rather than raw order IDs. Keep bearer tokens and sensitive payloads out of logs.

An API gateway provides a stable entry point for routing and shared API policies. Keep its responsibilities focused on decisions it can make reliably, while services retain business validation and resource authorization.

Define how proxies pass identity information and how callers can reach backend services. Preserve the public API's behavior as requests pass through proxies, and remember that a timeout can leave the outcome unknown. Operate gateway configuration with the same care as application code: bounded retries, tested failure behavior, redundant capacity, gradual rollout, and a working rollback path.