Design a rate limiter: a practical guide for developers

Introduction

Your API is getting hit hard. Traffic spikes, bot attacks, or a misbehaving client hammering your endpoints—any of these can bring a service down faster than you'd expect. The answer isn't to throw more hardware at the problem; it's to design a rate limiter: a practical guide for developers that sits between your clients and your expensive logic, saying "not now" when things get out of hand. A well-designed rate limiter protects your infrastructure, ensures fair access for legitimate users, and gives you breathing room to investigate issues without the entire platform melting.

We've built rate limiters across ride-hailing platforms, SaaS backends, and e-commerce checkouts. Each scenario taught us that placement, algorithm choice, and client communication matter as much as the implementation itself. This guide covers the decisions you need to make and the patterns that work in production.

Where it lives: Gateway or proxy layer

The first question isn't "how do I count requests?" It's "where do I count them?" The answer is almost always: before expensive work happens.

Rate limiting at the API gateway or reverse proxy (nginx, Kong, AWS API Gateway, or a custom proxy service) stops bad traffic at the perimeter. A request rejected at the gateway never reaches your application servers, never hits your database, never burns CPU cycles. If you're using NestJS, Next.js, or .NET, you can add rate limiting middleware, but you'll get better protection and efficiency by sitting one layer back.

In a typical architecture:

• Client request arrives
• Gateway checks rate limit (before routing)
• If rejected: return 429 immediately
• If allowed: forward to application, increment counter

This matters because you're protecting three things: your public endpoints, your backend resources, and your database. A single rate limit rule at the gateway can prevent a misconfigured mobile app from causing a cascade failure.

Real example: A travel platform's mobile client shipped with a bug that retried search requests every 500ms on network errors. The traffic spike hit the gateway first. A rate limit per IP blocked the client gracefully (returning 429) before the search query could reach the database. The team fixed the client, re-enabled traffic, and no one noticed.

Algorithms: Fixed window, sliding window, and token bucket

Not all rate limiters are created equal. The algorithm you choose affects fairness, burstiness, and complexity.

Fixed window: simple, but bursty

Divide time into fixed buckets (e.g., 1-minute windows). Count requests in the current window. When the window resets, the counter goes to zero.

Behavior: A user with a 100 request/minute limit can send 100 requests at 59 seconds, then 100 more at 60 seconds. That's 200 requests in 2 seconds—a burst at the window edge.

Implementation in Redis:

async function checkFixedWindow(userId: string, limit: number, windowSec: number): Promise {
  const key = `rate:${userId}`;
  const now = Math.floor(Date.now() / 1000 / windowSec);
  const windowKey = `${key}:${now}`;
  
  const count = await redis.incr(windowKey);
  if (count === 1) {
    await redis.expire(windowKey, windowSec + 1);
  }
  
  return count <= limit;
}

Pros: Dead simple. Single Redis key, single increment.

Cons: Allows uneven bursts at window boundaries.

Sliding window: smoother limits

Instead of discrete windows, imagine a sliding frame. For a 100 request/60-second limit, check: "How many requests happened in the past 60 seconds (from now)?" If fewer than 100, allow the request.

Implementation: Store request timestamps in a sorted set. Remove old entries. Count remaining ones.

async function checkSlidingWindow(userId: string, limit: number, windowSec: number): Promise {
  const key = `rate:slide:${userId}`;
  const now = Date.now();
  const windowStart = now - (windowSec * 1000);
  
  // Remove old entries
  await redis.zremrangebyscore(key, '-inf', windowStart);
  
  // Count requests in window
  const count = await redis.zcard(key);
  
  if (count < limit) {
    await redis.zadd(key, now, `${now}-${Math.random()}`);
    await redis.expire(key, windowSec + 1);
    return true;
  }
  
  return false;
}

Pros: No burst at window edges. Fair and predictable.

Cons: More Redis operations. Higher latency per request.

Token bucket: healthy bursts allowed

Imagine a bucket that fills with tokens at a constant rate (e.g., 10 tokens per second). Each request costs 1 token. If the bucket is full, excess tokens are discarded. This allows short bursts (up to bucket size) while smoothing demand over time.

Behavior: With a bucket size of 20 and a refill rate of 10 tokens/sec, a user can burst 20 requests immediately, then maintain 10 requests/sec indefinitely.

Implementation with Lua to ensure atomicity:

async function checkTokenBucket(userId: string, bucketSize: number, refillRate: number, windowSec: number = 1): Promise {
  const key = `rate:tokens:${userId}`;
  const now = Date.now();
  
  const script = `
    local key = KEYS[1]
    local now = tonumber(ARGV[1])
    local rate = tonumber(ARGV[2])
    local size = tonumber(ARGV[3])
    
    local data = redis.call('hmget', key, 'tokens', 'lastRefill')
    local tokens = tonumber(data[1]) or size
    local lastRefill = tonumber(data[2]) or now
    
    local timePassed = math.max(0, (now - lastRefill) / 1000)
    local tokensAdded = timePassed * rate
    tokens = math.min(size, tokens + tokensAdded)
    
    if tokens >= 1 then
      tokens = tokens - 1
      redis.call('hset', key, 'tokens', tokens, 'lastRefill', now)
      redis.call('expire', key, 3600)
      return 1
    end
    
    return 0
  `;
  
  const result = await redis.eval(script, 1, key, now, refillRate, bucketSize);
  return result === 1;
}

Pros: Allows healthy bursts. Smooth under sustained load. Industry standard (used by AWS, Stripe).

Cons: Slightly more complex logic. Requires careful timestamp handling across distributed clocks.

Distributed counters: keeping limits consistent

When your API runs on multiple servers, your rate limiter must see a unified view of traffic. This is where Redis shines.

Atomic increments matter. If server A increments a counter from 99 to 100 while server B does the same, you need a guarantee: one of them hits the limit, not both. Redis INCR is atomic. Multiple clients can increment the same key concurrently without race conditions.

For complex logic (like token bucket), Lua scripts are your friend. A Lua script runs on the Redis server as a single atomic operation. The entire script executes before any other client's commands are processed.

Example architecture:

For high-traffic systems, replicate Redis across multiple regions. Use local Redis instances in each region with periodic sync to a central store, or use Redis Cluster for automatic sharding. The key is: never let rate limit checks become a bottleneck.

Tell clients: 429 and rate limit headers

When you reject a request, the HTTP status code and response headers matter. They tell the client what happened and when to retry.

Return 429 Too Many Requests:

HTTP/1.1 429 Too Many Requests
Retry-After: 15
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1702500900

{
  "error": "rate_limit_exceeded",
  "message": "You have exceeded 100 requests per minute. Try again in 15 seconds."
}

Header meanings:

• Retry-After: Seconds to wait before retrying (or an HTTP date). Clients should honor this; SDKs parse it automatically.
• X-RateLimit-Limit: The limit for this endpoint (requests per window).
• X-RateLimit-Remaining: How many requests the user has left before hitting the limit.
• X-RateLimit-Reset: Unix timestamp when the limit resets.

Honest headers prevent cascading failures. A well-written client sees Retry-After: 30, backs off for 30 seconds, and retries. A client that guesses and retries every second just hammers you harder.

Middleware example (NestJS):

@Injectable()
export class RateLimitInterceptor implements NestInterceptor {
  async intercept(context: ExecutionContext, next: CallHandler): Promise> {
    const request = context.switchToHttp().getRequest();
    const response = context.switchToHttp().getResponse();
    const userId = request.user?.id || request.ip;
    
    const allowed = await checkTokenBucket(userId, 100, 10);
    
    if (!allowed) {
      response.status(429);
      response.set('Retry-After', '10');
      response.set('X-RateLimit-Limit', '100');
      response.set('X-RateLimit-Remaining', '0');
      throw new HttpException('Rate limit exceeded', HttpStatus.TOO_MANY_REQUESTS);
    }
    
    return next.handle();
  }
}

Common mistakes

Mistake 1: Rate limiting by IP only. Behind a corporate proxy or NAT, many users share one IP. You'll throttle legitimate traffic. Combine IP with user ID or API key if available.

Mistake 2: No grace period or exemptions. Internal services, critical workflows, or specific API keys sometimes need higher limits. Build a bypass mechanism or tiered limits.

Mistake 3: Rejecting without context. Returning 429 with no Retry-After header or error message leaves clients confused. Always explain what happened and when to try again.

Mistake 4: Assuming Redis is fast enough. For ultra-high throughput, even Redis latency adds up. Profile your P99 latencies. If rate limit checks become your bottleneck, consider caching or approximate counting.

Mistake 5: Ignoring clock skew. In distributed systems, server clocks drift. Token bucket logic depends on accurate timestamps. Use NTP and monitor clock offsets.

Key takeaways

• Place rate limiters at the gateway or proxy layer, before expensive operations. This protects your entire backend.
• Choose your algorithm based on fairness and burstiness tolerance. Token bucket is the industry default; fixed window is simplest; sliding window is fairest.
• Use Redis with atomic increments or Lua scripts to maintain consistent limits across multiple servers. Never rely on per-server counters.
• Always return 429 with Retry-After and rate limit headers. Help clients make intelligent retry decisions.
• Test under load. Profile rate limiter latency. A slow rate limiter becomes your new bottleneck.
• Monitor and alert on rate limit hit rates. Spikes reveal attacks, bugs, or misconfigured clients before they cause outages.