As a system grows from one service into many, clients can quickly end up with a messy job. They might have to call several backend services directly, handle authentication for each one, retry failed calls, deal with rate limits, and keep up with every backend change.

An API gateway is a controlled front door for client traffic. Clients call the gateway. The gateway checks the request, applies shared rules, sends the request to the right backend service, and returns the response.

The main idea is simple: keep the public API stable while backend services change behind it. The gateway handles common work such as authentication, rate limiting, request validation, and logging in one place, instead of copying that logic into every service.

A gateway is not always needed. A single application behind a load balancer can be enough. A gateway becomes useful when clients need one stable entry point in front of many services, many client types, or many shared rules.

This chapter covers what an API gateway does and where it fits.

Consider a commerce application. One product page might need product details from the catalog service, prices and discounts from the pricing service, inventory status from the inventory service, recommendations from a personalization service, and the user's cart state from the cart service.

If a mobile app calls all of those services directly, the app now knows too much about the backend. Every service split, rename, migration, authentication change, or protocol change can force a mobile app release.

That is especially painful for mobile and partner clients. Old app versions can stay in circulation for months. Partner API contracts often require advance notice before breaking changes.

An API gateway gives clients one stable API to call.

The gateway does not magically make the backend simple. It gives the system a clear front door where shared rules can be handled consistently.

Calling services directly can work in a small system. It becomes painful when the system has many services, clients, teams, and security rules.

Clients Learn Too Much About the Backend

A client should not need to know that checkout-service was split into order-service, payment-service, and fulfillment-service.

Internal service names and boundaries are implementation details. Public APIs should change more slowly than the backend design.

Shared Rules Get Duplicated

Without a gateway, each public service often grows its own version of the same front-door logic: authentication, authorization checks, rate limiting, CORS handling, request validation, API key handling, audit logs, and request IDs.

When every service implements those rules separately, they start to drift. One service accepts an expired token. Another logs sensitive fields. A third uses different rate limits. The system becomes harder to understand and harder to secure.

Clients Become Chatty

Every remote call has a cost. On mobile networks, one extra request may mean another secure connection setup, another timeout, another retry, or another half-loaded screen.

Gateways can reduce the number of calls a client has to make. For example, GET /mobile/home might return the data needed to render the home screen instead of forcing the app to coordinate seven backend calls.

Use this carefully. Combining data at the gateway is reasonable when it is part of the client-facing API. Business rules still belong in the services that own the data.

Protocols Do Not Always Match

External clients usually want stable HTTPS APIs. Internal services may use HTTP, gRPC, message queues, event streams, private provider APIs, or model-serving APIs.

A gateway can translate between the public API and the internal way a service is called. But it should not change behavior that clients depend on. For example, a streaming AI response should keep streaming instead of buffering the whole answer and making the user wait.

Different products use the term "API gateway" differently. Some gateways are lightweight reverse proxies. Some are full API management platforms. Some are cloud-managed services. Common examples include Envoy, NGINX, HAProxy, Kong, Traefik, Apigee, AWS API Gateway, Azure API Management, and custom edge services.

The responsibilities are usually drawn from the same set.

The gateway is a control point. It should be boring, predictable, and easy to monitor.

These components often sit near each other, but they solve different problems.

A real production path may use several of these together:

The API gateway is the client-facing API boundary in that chain. The service mesh is usually for service-to-service traffic inside the system. A load balancer focuses on spreading traffic across healthy instances.

Kubernetes Ingress is still widely used, but Gateway API is the newer Kubernetes model for traffic routing. It is related to gateway concepts, but it is not the same thing as every product called an API gateway.

A gateway handles work that many client requests share, so backend services do not each have to reimplement it. This work belongs near the front door, where it can be applied consistently.

Routing

Routing maps a public request to a backend destination.

Simple routing uses path and method:

More advanced routing can use other signals. It can send api.example.com and partner.example.com to different backends, or route based on a header such as X-Client-Version: 12.

It can route a tenant to its assigned region, send a small slice of traffic, such as 5%, to a new version, or send streaming requests only to backends that support streaming.

Routing rules should be reviewed like code. A bad route change can break production as quickly as a bad deploy.

Authentication and Broad Authorization

The gateway is usually the first place to verify who is calling from outside the system.

It can validate JWTs, including who issued the token, who the token is for, and whether it has expired. It can integrate with OAuth2 and OpenID Connect. It can validate API keys for partner or server-to-server clients and require mutual TLS for high-trust machine clients. It can also pass verified identity information down to backend services.

The gateway can reject obviously unauthorized traffic early. It should not become the only authorization layer.

Backend services still need to enforce business permissions. The gateway may know that a user is signed in and can call the orders API. The order service must still decide whether that user can read order 12345.

Rate Limiting and Quotas

Rate limiting protects backend capacity and makes abuse harder.

Useful limits are rarely just "100 requests per minute per IP." Production systems often limit by user, organization or tenant, API key, endpoint, and region. AI workloads may also limit by model, provider, or token budget. Write-heavy operations such as checkout or password reset often get tighter limits.

For AI systems, request count alone is often the wrong unit. One request with a 200,000-token context can cost more than hundreds of small metadata requests. Gateways or nearby policy services may enforce budgets using estimated tokens, output limits, model class, tenant tier, or daily spend.

When a request is limited, return 429 Too Many Requests with enough information for a well-behaved client to slow down and retry later.

Request Validation

Gateways should reject clearly invalid requests before they hit backend services.

Validation can check for required headers, an acceptable content type, and a reasonable body size. It can check that the request body has the expected shape, allow only known methods, and require a recognized API version. It can also enforce a maximum upload size and a maximum prompt size.

Gateway validation is not a replacement for validation inside the service. Treat it as an early filter that protects capacity and gives clients faster feedback.

Transformation and Protocol Translation

Gateways can adapt the public API to the internal service API. They might convert a public JSON request into an internal gRPC call, rename fields during an API version change, remove internal-only fields before returning a partner response, standardize backend errors into one public error format, or preserve Server-Sent Events and streaming HTTP responses for chat completions.

Transformation is useful when it protects the public API. It becomes dangerous when it turns into business logic. If the gateway starts calculating discounts, choosing fulfillment rules, or deciding fraud outcomes, the responsibility is in the wrong place.

Caching

Gateways can cache responses for safe, repeatable reads.

Good candidates are public product metadata, feature flag bootstrap responses, static configuration, anonymous catalog pages, and expensive read-only partner endpoints. Poor candidates are user-specific secrets, checkout state, payment status without strict cache rules, personalized AI responses, and anything where stale data could create a correctness or privacy problem.

Caching at the gateway needs clear cache keys, TTLs, invalidation rules, and privacy controls. A missing tenant ID or authorization value in the cache key can leak data across users.

Observability

Gateways see the front door of the system, so they are valuable places to collect debugging information. They should record request counts by route, method, status code, tenant, and client type. They should also record latency percentiles, upstream error rates, rate-limit decisions, authentication failures, request and trace IDs, and request size or token usage where appropriate.

Do not log secrets, authorization headers, raw payment data, or full AI prompts by default. Logging is part of security, not an afterthought.

A typical request goes through the gateway in this order:

For a food delivery app, POST /orders might look like this:

The gateway did not own the checkout workflow. Long-running or business-critical coordination belongs in backend services, workflow engines, or orchestration layers. The gateway should stay focused on front-door responsibilities.

Once a gateway is in place, a few decisions shape how well it fits the system. These choices are easier to get right early than to undo later.

One Gateway or Many?

A single shared gateway is simple to start with. It gives all clients the same entry point and keeps platform policy centralized.

As the system grows, many teams split gateways by audience or risk. They might run separate gateways for the public consumer API, the partner API, and the internal admin API, plus a mobile BFF, an AI tool or agent API, and a regional gateway where geography matters.

Multiple gateways can limit how much damage one bad change can cause and make ownership clearer. They also add more systems to run. Choose the split based on API ownership, security needs, latency goals, and how often each API changes.

Gateway or BFF?

An API gateway is a shared front door. A Backend for Frontend, often called a BFF, is a backend built for one client experience.

Use a gateway for common front-door concerns: authentication, routing, rate limits, TLS, logging, and broad policy.

Use a BFF when a client needs its own API shape, data-combining strategy, caching behavior, or release pace. Do not keep adding mobile-specific and web-specific branches to a shared gateway until it becomes an application with no clear owner.

Managed Service or Self-Hosted Gateway?

Managed gateways reduce operational work. They can be a good fit when the traffic pattern matches the provider's model and the platform team wants less infrastructure to run.

Self-hosted gateways give more control over latency, networking, plugins, custom rules, streaming behavior, and deployment layout. They also require teams to own upgrades, capacity planning, security patches, and incident response.

The wrong choice is the one nobody operates well.

Synchronous, Streaming, and Long-Lived Traffic

Not all API traffic is short HTTP request-response traffic.

Gateways may also need to handle WebSockets, Server-Sent Events, gRPC streams, file uploads, long polling, and token-streaming AI responses.

These workloads need careful timeout, buffering, retry, and connection-draining settings. A gateway that works well for small JSON responses can still break streaming responses by buffering them, closing idle connections too early, or retrying requests that are not safe to repeat.

Gateways fail in predictable ways. Most problems happen when the gateway takes on work that belongs elsewhere, or when teams forget that the gateway is critical production infrastructure.

Putting Business Logic in the Gateway

The gateway should enforce front-door rules and adapt API responses when needed. It should not own business rules.

If checkout correctness depends on gateway code, the design is fragile. A batch job, internal admin tool, or future service may bypass the gateway and miss those same rules.

Making the Gateway a Single Point of Failure

Because all client traffic passes through the gateway, it must be deployed like critical infrastructure. Run multiple instances across multiple zones. Use health checks and autoscaling. Roll out configuration safely. Keep rollback fast, because a bad gateway change can break the whole API.

For security decisions, the gateway should fail closed. If it cannot verify a token, it should reject the request. For traffic decisions, it should fail predictably so clients get clear errors instead of random behavior.

Using Retries Without Idempotency

Gateways can retry failed backend calls, but retries are not harmless.

Retrying GET /products is usually safe. Retrying POST /orders can create duplicate orders unless the operation uses idempotency keys or service-side duplicate detection.

Retries need timeouts, limits, and clear rules about which methods and status codes are safe to retry.

Logging Sensitive Data

Gateway logs are tempting because they sit in one central place. That also makes them dangerous.

Do not log access tokens, API keys, passwords, payment fields, private customer data, or full AI prompts unless there is a specific approved reason. Prefer masking sensitive fields, hashing values when useful, sampling noisy logs, and logging only allowed fields.

Treating Gateway Configuration as "Just Config"

Gateway configuration is production code. It changes routing, authentication, limits, headers, and client behavior.

Use reviews, tests, staged rollout, config validation, and ownership. A one-line route change can take down an API.

Do not add a gateway just because the architecture diagram feels incomplete.

You may not need one when a single application already serves the public API cleanly, or when a load balancer covers the traffic needs on its own. The same is true when there is only one trusted client, when the team cannot run another critical component well, or when the gateway would only forward every request unchanged.

In those cases, keep the system simpler. Add a gateway when it solves a real boundary problem: multiple clients, multiple services, repeated front-door rules, mismatched protocols, API stability, or stronger traffic control.

An API gateway is the client-facing front door for backend services. It handles shared concerns such as routing, authentication, rate limiting, validation, logging, metrics, and traffic control.

It should not become the owner of business logic or long-running workflows. That work belongs in the services behind it.

It can appear in the same traffic path as a load balancer, Kubernetes Ingress, and service mesh, but it plays a different role from each one.

Gateway design is mostly a question of ownership: who owns the public API, who owns the shared rules, and who keeps the request path healthy when something fails.