API gateway pattern: a practical Design Patterns guide

When you're building a distributed system, the last thing you want is for clients to know about every single backend service. The API gateway pattern solves this by creating a single entry point that routes requests to the right services, handles cross-cutting concerns, and shields your clients from your internal architecture. Over 5+ years building production platforms at scale, I've seen teams either get this right and sleep peacefully, or get it wrong and spend weeks debugging cascading failures across services.

Why you need an API gateway

Imagine you're running an e-commerce platform with separate services for products, orders, payments, and inventory. Without a gateway, your mobile app would need to know about all four services, handle retries independently, manage authentication separately for each one, and deal with network issues on its own. That's not maintainable. An API gateway centralizes these concerns so your clients stay simple and your backend remains flexible.

In practice, I've deployed gateways in front of ride-hailing platforms, trading systems, and SaaS products. The pattern consistently gives you three immediate wins: simplified client code, easier deployment (you can add services without client changes), and a central place to enforce security and reliability rules.

Responsibilities — Routing, auth, rate limiting and TLS termination

A well-designed API gateway owns exactly four core responsibilities, and nailing these prevents most production issues.

Routing

The gateway inspects the incoming request path and headers, then forwards it to the appropriate backend service. If a request comes in for /api/orders/*, it goes to the orders service. If it's /api/products/*, it goes to the products service. This is straightforward, but routing rules grow complex fast—you might need path prefixes, hostname-based routing, or even weight-based traffic splitting for canary deployments.

With YARP (Yet Another Reverse Proxy), I typically configure routes declaratively:

{
  "ReverseProxy": {
    "Routes": {
      "orders": {
        "ClusterId": "orders-cluster",
        "Match": { "Path": "/api/orders/{**catch-all}" }
      },
      "products": {
        "ClusterId": "products-cluster",
        "Match": { "Path": "/api/products/{**catch-all}" }
      }
    },
    "Clusters": {
      "orders-cluster": {
        "Destinations": {
          "orders-1": { "Address": "http://orders-service:5000" }
        }
      }
    }
  }
}

Authentication and authorization

Never let an unauthenticated request reach your backend services. The gateway validates JWT tokens, API keys, or OAuth tokens before forwarding anything downstream. This is where you decode the token, verify the signature, check expiration, and extract claims. Backend services can then trust the user identity without re-validating.

I've built this as middleware in NestJS-based gateways:

@Injectable()
export class AuthMiddleware implements NestMiddleware {
  constructor(private jwtService: JwtService) {}

  use(req: Request, res: Response, next: NextFunction) {
    const token = req.headers.authorization?.split(' ')[1];
    
    if (!token) {
      return res.status(401).json({ error: 'Unauthorized' });
    }

    try {
      const decoded = this.jwtService.verify(token);
      req.user = decoded;
      next();
    } catch (err) {
      res.status(401).json({ error: 'Invalid token' });
    }
  }
}

Rate limiting

A single misbehaving client shouldn't take down your entire platform. The gateway enforces rate limits per user, per IP, or per API key. When a client exceeds the limit, the gateway rejects the request before it ever reaches your backend, saving compute and database load.

Nginx handles this elegantly with the limit_req module:

limit_req_zone $binary_remote_addr zone=general:10m rate=10r/s;
limit_req_zone $http_x_api_key zone=by_key:10m rate=100r/s;

server {
  location /api/ {
    limit_req zone=general burst=20;
    limit_req zone=by_key burst=50;
    proxy_pass http://backend;
  }
}

TLS termination

All external traffic should be encrypted. Rather than having each backend service handle TLS certificates, the gateway terminates TLS—it decrypts incoming HTTPS traffic, processes it, and forwards unencrypted requests internally (over private networks). This centralizes certificate management and reduces CPU overhead on backend services.

In production, I always use a load balancer or reverse proxy like Nginx or AWS Application Load Balancer to handle TLS. Your certificate renewal, cipher suite updates, and protocol upgrades happen in one place.

Request aggregation — Combine multiple service calls into one response

Sometimes a single client request requires data from multiple backend services. For instance, an e-commerce order details page needs the order metadata, the items in it, the customer profile, and the shipping status. Without aggregation, the client would make four separate calls with four round trips and four failure points.

The gateway can orchestrate these calls, wait for all responses, combine them, and return one unified response. This is especially powerful for mobile clients on slow networks.

Here's a practical example using NestJS:

@Get('/orders/:id/details')
async getOrderDetails(@Param('id') orderId: string) {
  const [order, items, customer, shipping] = await Promise.all([
    this.httpService.get(`http://orders-service/orders/${orderId}`).toPromise(),
    this.httpService.get(`http://items-service/orders/${orderId}/items`).toPromise(),
    this.httpService.get(`http://customers-service/${order.customerId}`).toPromise(),
    this.httpService.get(`http://shipping-service/orders/${orderId}`).toPromise(),
  ]);

  return {
    order: order.data,
    items: items.data,
    customer: customer.data,
    shipping: shipping.data,
  };
}

I've used this pattern heavily in travel platforms where a single search result combines flight availability, pricing, inventory, and review data from separate services. The key is keeping the logic simple: fan out the calls in parallel, handle timeouts gracefully, and return partial data if some services are slow but not critical.

Don't put business logic here — Gateways should stay thin

This is where teams often stumble. The gateway feels like a convenient place to add features: discount calculations, business rule validations, complex transformations. Don't do it. A fat gateway becomes a bottleneck and a maintenance nightmare.

The gateway should strictly own cross-cutting concerns: routing, auth, rate limiting, request/response transformation, and logging. Everything else belongs in the service layer where it's properly tested, versioned, and owned by a team.

I've debugged production incidents where the gateway was running 10+ years of legacy business logic that nobody understood. When the platform scaled and we needed to update the logic, we couldn't add a second gateway instance without replicating and synchronizing that logic. We ended up rewriting it and moving everything back into services.

The rule I follow: if the logic needs to change independently of the gateway's infrastructure, it belongs in a service.

Options — Nginx, YARP, Kong, AWS API Gateway

Nginx

Lightweight, battle-tested, and simple. Perfect for basic routing, TLS termination, and rate limiting. I use it for everything that doesn't require dynamic business logic. Configuration is declarative, performance is exceptional, and operational overhead is minimal. Downside: extending it requires Lua scripting or recompiling, which gets complex fast.

YARP (Yet Another Reverse Proxy)

Built by Microsoft in .NET, YARP is the middle ground. It's a library you embed in an ASP.NET Core application, so you get full C# power while keeping most of the reverse proxy simplicity. I use it when I need routing, auth, and light orchestration without building a full gateway service from scratch. Configuration is JSON or code-based, and extending it is just writing C#.

Kong

An open-source API gateway built on Nginx and Lua. It adds a management API, plugin ecosystem, and dashboard on top of Nginx. Kong is enterprise-grade: it handles complex routing, authentication plugins (OAuth, JWT, LDAP), rate limiting, request transformation, and more. I've deployed it in teams that needed managed gateways with audit trails and compliance features. Trade-off: it's heavier than plain Nginx and introduces operational complexity.

AWS API Gateway

Fully managed, serverless, and scales infinitely. You define routes, authentication, rate limiting, and request/response transformations through the console or CloudFormation. It integrates seamlessly with Lambda, DynamoDB, and other AWS services. Perfect for teams without operations expertise or for highly variable traffic patterns. Cost scales with usage, which can get expensive at high throughput.

In my experience, choose Nginx for simplicity, YARP if you're already in the .NET ecosystem, Kong if you need an enterprise feature set, and AWS API Gateway if you want zero operations overhead.

Common mistakes

• Making the gateway stateful: Don't store session data or cache in a single gateway instance. You can't scale horizontally. Use external cache (Redis) or make the gateway stateless.
• Mixing infrastructure and business logic: Gateways break when they contain complex business rules. Keep them thin and move logic to services.
• Not handling downstream failures gracefully: If a service is slow or down, the gateway times out and returns 504 to the client. Implement circuit breakers, fallbacks, and bulkheads to fail fast and protect upstream.
• Ignoring observability: You can't debug production issues without logging, tracing, and metrics in the gateway. Instrument it heavily from day one.
• Over-engineering routing: Start simple. As complexity grows, move advanced routing logic to a service mesh (Istio, Linkerd) instead of the gateway.

Key takeaways

• An API gateway centralizes routing, authentication, rate limiting, and TLS termination, giving you a single control point for all external traffic.
• Use request aggregation to combine calls to multiple backend services and reduce client round trips, but keep the logic simple.
• Keep the gateway thin. Cross-cutting concerns belong there; business logic belongs in services.
• Choose your tool based on your team's expertise and scale: Nginx for simplicity, YARP for .NET, Kong for enterprise features, AWS API Gateway for serverless.
• Design the gateway to be stateless, observable, and resilient to downstream failures.