Rate limiter patterns: a practical Design Patterns guide

When your API is hit by a traffic spike, a misconfigured client, or worse—an attacker—your system can become unresponsive to legitimate users. Rate limiter patterns: a practical Design Patterns guide gives us proven ways to control traffic flow and protect infrastructure. In this guide, we'll explore four key strategies, from simple to sophisticated, and show you when to use each one.

Why rate limiting matters

Rate limiting isn't just about security; it's about reliability. Whether you're protecting a payment processor, a search engine, or a public API, uncontrolled traffic can degrade service for everyone. By implementing the right pattern, we prevent cascading failures, keep costs predictable, and maintain fair access for all users.

Fixed window: simple counters per time window

The fixed window approach divides time into fixed intervals—say, 60-second buckets—and counts requests within each. Once the counter exceeds the limit, we reject further requests until the next window starts. It's the simplest strategy and works well for straightforward rate limiting scenarios.

Pros: Easy to implement and understand. Low memory overhead. Fast calculations.

Cons: Allows traffic bursts at window boundaries. If your limit is 100 requests per minute and a user sends 100 requests at 59 seconds and another 100 at 61 seconds, they've exceeded their true rate.

Here's a basic implementation:

class FixedWindowLimiter {
  private counters: Map<string, { count: number; resetTime: number }> = new Map();
  private windowSizeMs: number;
  private maxRequests: number;

  constructor(maxRequests: number, windowSizeMs: number) {
    this.maxRequests = maxRequests;
    this.windowSizeMs = windowSizeMs;
  }

  isAllowed(userId: string): boolean {
    const now = Date.now();
    const key = userId;
    const entry = this.counters.get(key);

    if (!entry || now >= entry.resetTime) {
      // New window
      this.counters.set(key, {
        count: 1,
        resetTime: now + this.windowSizeMs,
      });
      return true;
    }

    if (entry.count < this.maxRequests) {
      entry.count++;
      return true;
    }

    return false;
  }
}

Use fixed window when you need simplicity and your traffic patterns are predictable, or when you're okay with occasional bursts.

Sliding window: smoother limits across window boundaries

Sliding window improves on fixed window by tracking requests within a rolling time period. Instead of discrete buckets, imagine a continuous window that moves forward. We keep timestamps of recent requests and reject any new request if it would create too many requests within the last interval.

Pros: Prevents boundary bursts. More fair rate limiting. Requests that arrive near the window edge are handled smoothly.

Cons: Higher memory overhead (we store individual request timestamps). More complex logic.

Here's a practical implementation:

class SlidingWindowLimiter {
  private requestTimestamps: Map<string, number[]> = new Map();
  private windowSizeMs: number;
  private maxRequests: number;

  constructor(maxRequests: number, windowSizeMs: number) {
    this.maxRequests = maxRequests;
    this.windowSizeMs = windowSizeMs;
  }

  isAllowed(userId: string): boolean {
    const now = Date.now();
    const key = userId;

    if (!this.requestTimestamps.has(key)) {
      this.requestTimestamps.set(key, [now]);
      return true;
    }

    const timestamps = this.requestTimestamps.get(key)!;

    // Remove timestamps outside the sliding window
    const cutoff = now - this.windowSizeMs;
    const validTimestamps = timestamps.filter((ts) => ts > cutoff);

    if (validTimestamps.length < this.maxRequests) {
      validTimestamps.push(now);
      this.requestTimestamps.set(key, validTimestamps);
      return true;
    }

    return false;
  }
}

Sliding window is ideal for APIs where you want strict fairness and smoother rate limiting behavior.

Token bucket: allows bursts within a sustained rate

Token bucket treats rate limiting as a bucket that fills at a constant rate. Each request consumes one token; if no tokens remain, the request is rejected. Between requests, tokens accumulate up to a maximum. This pattern elegantly handles both burst traffic and sustained rates.

Pros: Allows controlled bursts. Flexible and predictable. Fair over the long term. Commonly used in production systems.

Cons: Slightly more complex state management. Need to track the last refill time accurately.

Real-world scenario: You want to allow users to make up to 10 requests per second, but also allow bursts of up to 50 requests once per hour. Token bucket handles this naturally.

class TokenBucketLimiter {
  private buckets: Map<string, { tokens: number; lastRefillTime: number }> = new Map();
  private capacity: number;
  private refillRate: number; // tokens per second
  private refillIntervalMs: number = 1000;

  constructor(capacity: number, refillRate: number) {
    this.capacity = capacity;
    this.refillRate = refillRate;
  }

  isAllowed(userId: string, tokensNeeded: number = 1): boolean {
    const now = Date.now();
    const key = userId;

    if (!this.buckets.has(key)) {
      this.buckets.set(key, {
        tokens: this.capacity,
        lastRefillTime: now,
      });
    }

    const bucket = this.buckets.get(key)!;
    const timeSinceLastRefill = now - bucket.lastRefillTime;
    const tokensToAdd =
      (timeSinceLastRefill / this.refillIntervalMs) * this.refillRate;

    bucket.tokens = Math.min(
      this.capacity,
      bucket.tokens + tokensToAdd
    );
    bucket.lastRefillTime = now;

    if (bucket.tokens >= tokensNeeded) {
      bucket.tokens -= tokensNeeded;
      return true;
    }

    return false;
  }
}

Token bucket is our go-to for production APIs where we need to balance fairness with allowing occasional traffic spikes.

Distributed limits: Redis counters shared across instances

In a distributed system with multiple servers, in-memory rate limiters don't work—each instance tracks its own counters independently, allowing users to multiply their rate by the number of servers. Distributed rate limiting centralizes the counter in a shared store, typically Redis, so every instance enforces the same limit.

Pros: Enforces limits consistently across all servers. Scalable to any number of instances. Synchronized state.

Cons: Network latency adds overhead. Redis becomes a critical dependency. May need clustering for high availability.

Here's a Redis-backed token bucket implementation:

import Redis from "ioredis";

class DistributedTokenBucketLimiter {
  private redis: Redis;
  private capacity: number;
  private refillRate: number;

  constructor(redis: Redis, capacity: number, refillRate: number) {
    this.redis = redis;
    this.capacity = capacity;
    this.refillRate = refillRate;
  }

  async isAllowed(userId: string, tokensNeeded: number = 1): Promise<boolean> {
    const key = `rate_limit:${userId}`;
    const now = Date.now();

    const script = `
      local bucket = redis.call('HGETALL', KEYS[1])
      local tokens = tonumber(bucket[2]) or ARGV[3]
      local lastRefill = tonumber(bucket[4]) or ARGV[4]
      
      local timeSinceRefill = (ARGV[1] - lastRefill) / 1000
      local tokensToAdd = timeSinceRefill * ARGV[2]
      tokens = math.min(ARGV[3], tokens + tokensToAdd)
      
      if tokens >= ARGV[5] then
        tokens = tokens - ARGV[5]
        redis.call('HSET', KEYS[1], 'tokens', tokens, 'lastRefill', ARGV[1])
        redis.call('EXPIRE', KEYS[1], 3600)
        return 1
      end
      
      return 0
    `;

    const result = await this.redis.eval(script, 1, key, now, this.refillRate, this.capacity, now, tokensNeeded);
    return result === 1;
  }
}

We use Lua scripting here because it ensures atomic operations—the check and update happen without race conditions. This is critical in distributed systems where multiple servers might evaluate the same user's limit simultaneously.

For a load-balanced e-commerce API, distributed rate limiting prevents checkout abuse. Without it, an attacker could hit checkout endpoints on three separate servers 33 times each, bypassing a per-server 100-request limit.

Common mistakes

• Confusing local and distributed limits: Deploying a local in-memory limiter in a scaled environment is a security issue, not a feature.
• Ignoring clock skew: In distributed systems, server clocks drift. Use Redis timestamps or network time protocol to avoid off-by-one rejections.
• Not cleaning up old state: Without TTLs or cleanup, your rate limiter's memory grows indefinitely. Always set expiration times on old user entries.
• Failing silently on Redis outages: Decide upfront: do you fail open (allow all traffic) or fail closed (deny all traffic) if Redis is unavailable? Document it.
• Setting limits too tight: Legitimate spikes happen. A 1-request-per-second limit might be too strict for a mobile app with retry logic.

Key takeaways

• Fixed window is simple but allows bursts at boundaries; use it for straightforward scenarios.
• Sliding window smooths out boundary effects and is fairer over short timescales.
• Token bucket balances burst capacity with sustained rates and is preferred in production for flexible traffic patterns.
• Distributed (Redis-backed) rate limiting is essential in multi-instance systems; use Lua scripts to ensure atomicity.
• Always set TTLs on state, handle failures explicitly, and verify limits under load before going live.
• Choose your pattern based on traffic patterns, fairness requirements, and infrastructure complexity—there's no one-size-fits-all solution.