Backpressure in real-time systems in Kafka & WebSockets

You've scaled your trading platform to handle 10,000 concurrent WebSocket connections, but traders start reporting missing price updates. Your logs show memory climbing to 16GB, and within hours, the server crashes. This is backpressure in real-time systems—when producers flood faster than consumers can drain, buffers overflow, and the entire pipeline collapses.

After 72 days of building Kafka and WebSocket patterns, I've learned that backpressure isn't optional infrastructure; it's survival. In this post, I'll walk you through diagnosing backpressure, implementing bounded buffers, choosing when to drop data, and disconnecting misbehaving clients before they take your system down.

Symptoms: How backpressure kills systems

Backpressure shows up in predictable ways. Your memory usage creeps up steadily—not a memory leak, but unbounded queue growth. A single slow client on a 10 Mbps connection can't keep up with 100 price updates per second from your Kafka cluster. Those messages stack in memory waiting to be written.

The second warning sign is message latency. New traders connect and see prices from 5 seconds ago. The system isn't down, but it's useless. Third, and most dramatic, one morning your monitoring alerts fire: the server is OOM-killed, and all 10,000 connections drop at once.

I hit this on a ride-hailing platform where driver location updates came from Kafka at 50,000 events/sec, but mobile clients (often on 3G in traffic) could only accept 10 updates/sec. Without backpressure handling, we'd accumulate 40,000 events/sec per slow client. Multiply that by 100 slow connections, and the server would freeze within minutes.

The root cause is simple: asynchronous systems decouple producers and consumers, which sounds great until one side can't keep pace. Kafka gives you at-least-once delivery; WebSocket connections don't have flow control. So the memory in between—your message queue per connection—absorbs the difference until it explodes.

Bounded buffers: Queue depth as a firewall

The first defense is a bounded buffer. Instead of letting messages accumulate infinitely, cap the queue size per WebSocket connection. Here's the principle: if a client can't keep up, fail fast rather than fail catastrophically.

I typically set a buffer limit to 2–5 seconds of peak throughput for that client type. A price feed connection handling 100 updates/sec gets a buffer of 200–500 messages. A location update stream handling 1–2 updates/sec gets 5–10.

Here's a practical implementation using NestJS with a bounded queue:

import { WebSocketGateway, WebSocketServer, OnGatewayConnection } from '@nestjs/websockets';
import { Server, Socket } from 'socket.io';

interface QueuedMessage {
  type: string;
  payload: unknown;
  timestamp: number;
}

interface ClientState {
  queue: QueuedMessage[];
  maxQueueSize: number;
  droppedCount: number;
}

@WebSocketGateway()
export class PriceGateway implements OnGatewayConnection {
  @WebSocketServer() server: Server;
  private clientStates = new Map();

  handleConnection(socket: Socket) {
    this.clientStates.set(socket.id, {
      queue: [],
      maxQueueSize: 500, // Bounded buffer
      droppedCount: 0,
    });
  }

  handleDisconnect(socket: Socket) {
    this.clientStates.delete(socket.id);
  }

  sendPriceUpdate(clientId: string, price: { symbol: string; value: number }) {
    const state = this.clientStates.get(clientId);
    if (!state) return;

    const message: QueuedMessage = {
      type: 'price_update',
      payload: price,
      timestamp: Date.now(),
    };

    // Check buffer fullness
    if (state.queue.length >= state.maxQueueSize) {
      state.droppedCount++;
      console.warn(`Queue full for ${clientId}, dropped message (total: ${state.droppedCount})`);
      return;
    }

    state.queue.push(message);
    this.flushQueue(clientId);
  }

  private flushQueue(clientId: string) {
    const state = this.clientStates.get(clientId);
    if (!state || state.queue.length === 0) return;

    const socket = this.server.sockets.sockets.get(clientId);
    if (!socket) {
      state.queue = []; // Client disconnected
      return;
    }

    // Non-blocking send; if socket write buffer is full, we've already rejected at app level
    while (state.queue.length > 0) {
      const msg = state.queue.shift();
      socket.emit('price', msg.payload);
    }
  }
}

The key insight: when the queue hits its limit, drop the message immediately with a counter. Clients that are too slow aren't "special"—they need to deal with packet loss, like any network protocol. This prevents one slow client from poisoning the whole system.

Drop or conflate: Choosing what matters

Not all backpressure deserves the same response. For idempotent, state-based data (prices, locations, inventory), you can drop old messages. For transactional, event-based data (orders, payments, audit logs), you must never drop.

Two strategies emerge: dropping and conflating.

Dropping is brutal but fast. A new price update arrives while two old ones sit in the queue? Discard the old ones. The client gets the latest state, not a complete history. This works for real-time streams where the newest value is the only one that matters.

Conflating

Here's a conflation example for a ride-hailing system:

interface ClientState {
  messagesByType: Map; // Key: message type, value: latest message
  maxTypes: number;
  totalDropped: number;
}

sendLocationUpdate(clientId: string, location: { driverId: string; lat: number; lng: number; timestamp: number }) {
  const state = this.clientStates.get(clientId);
  if (!state) return;

  const messageKey = `location:${location.driverId}`;

  // If we already have this driver's location queued, replace it (conflate)
  if (state.messagesByType.has(messageKey)) {
    state.totalDropped++;
  }

  state.messagesByType.set(messageKey, {
    type: 'location',
    payload: location,
    timestamp: Date.now(),
  });

  // Still enforce a limit on distinct message types to prevent unbounded growth
  if (state.messagesByType.size > 1000) {
    const oldest = Array.from(state.messagesByType.values()).reduce((a, b) =>
      a.timestamp < b.timestamp ? a : b
    );
    state.messagesByType.delete(
      Array.from(state.messagesByType.entries()).find(([_, v]) => v === oldest)?.[0]!
    );
    state.totalDropped++;
  }

  this.flushQueue(clientId);
}

With conflation, you trade message fidelity for throughput. You won't deliver every event, but clients always see the latest state without queue explosion. This is acceptable for state synchronization; never use it for event ordering.

Disconnect slow clients: The nuclear option

Some clients are just broken. A mobile app crashes but keeps the WebSocket alive. A proxy hangs. A robot hammers your API without respecting backpressure signals. One misconfigured client should not drag down 10,000 good ones.

After a configurable timeout with a full queue, disconnect the client. This is harsh but effective. It frees memory, prevents memory leaks, and signals to the client that something is wrong.

Set reasonable thresholds based on your use case:

• Real-time price feeds: 5–10 second timeout with a full queue.
• Location streams: 10–30 second timeout (consider poor networks).
• Chat/messaging: 60+ second timeout (users on metro trains).

Implementation:

interface ClientState {
  queue: QueuedMessage[];
  maxQueueSize: number;
  droppedCount: number;
  queueFullSince: number | null;
  queueFullTimeoutMs: number;
}

handleConnection(socket: Socket) {
  this.clientStates.set(socket.id, {
    queue: [],
    maxQueueSize: 500,
    droppedCount: 0,
    queueFullSince: null,
    queueFullTimeoutMs: 10000, // 10 seconds
  });
}

sendPriceUpdate(clientId: string, price: { symbol: string; value: number }) {
  const state = this.clientStates.get(clientId);
  const socket = this.server.sockets.sockets.get(clientId);
  if (!state || !socket) return;

  const message: QueuedMessage = {
    type: 'price_update',
    payload: price,
    timestamp: Date.now(),
  };

  if (state.queue.length >= state.maxQueueSize) {
    // Queue is full
    if (state.queueFullSince === null) {
      state.queueFullSince = Date.now();
      console.warn(`Queue full for ${clientId}, starting timeout countdown`);
    } else if (Date.now() - state.queueFullSince > state.queueFullTimeoutMs) {
      // Timeout exceeded; disconnect the client
      console.error(`Client ${clientId} exceeded backpressure timeout. Disconnecting.`);
      socket.disconnect(true);
      this.clientStates.delete(clientId);
      return;
    }
    state.droppedCount++;
    return;
  }

  // Queue has space
  if (state.queueFullSince !== null) {
    console.info(`Queue resumed for ${clientId}, cleared timeout`);
    state.queueFullSince = null;
  }

  state.queue.push(message);
  this.flushQueue(clientId);
}

The logic: if a client's queue stays full for longer than the threshold, they're not draining messages. Disconnect them. Reconnection signals the client to fix whatever is wrong (network, app crash, bug).

In production, I pair this with a metric: disconnection counts per hour. If 10% of connections disconnect due to backpressure, something is broken upstream (e.g., Kafka is slow, or your message rate is unsustainable). This metric becomes an SLA violation you must investigate.

Common mistakes

Mistake 1: Unbounded queues with hope. Developers often assume "we'll just add more RAM" or "clients will eventually catch up." They won't. Memory is finite, and slow clients are a permanent problem in any real-time system. Bound the queue from day one.

Mistake 2: Treating all message types the same. Orders and prices are not interchangeable. Orders are transactional; prices are state. If you conflate orders, you lose data integrity. If you hold every price forever, you OOM. Know which is which.

Mistake 3: Disconnecting without warnings. Clients hit by a sudden disconnect may not understand why. Send a backpressure warning message before the 10-second timeout starts. Give them time to adapt (reduce subscription size, implement exponential backoff on reconnect).

Mistake 4: Ignoring slow readers on the Kafka side. Backpressure isn't just a WebSocket problem. If your Kafka consumer is slow, messages accumulate in the broker's fetch buffer. Use Kafka's fetch.min.bytes, fetch.max.wait.ms, and consumer lag monitoring to catch this early.

Mistake 5: No observability. If you can't see queue depth, dropped counts, and disconnect reasons per client, you're flying blind. Instrument everything: max queue depth per minute, percentile latencies, disconnect counts by reason.

Key takeaways

• Backpressure is inevitable in asynchronous systems. Producers and consumers operate at different speeds. Accept it and design for it.
• Bound your buffers. Set a per-connection queue limit based on peak throughput and acceptable latency. When the queue hits the limit, drop or disconnect, not queue indefinitely.
• Conflate where appropriate. For state-based data (prices, locations), keeping only the latest message per type prevents memory explosion while preserving correctness.
• Disconnect slow clients with a timeout. After a queue is full for N seconds, that client is broken. Disconnecting them protects the rest of your system and signals the client to fix the underlying issue.
• Distinguish transactional from state-based data. Never drop orders or payments. Drop prices without hesitation. Your backpressure strategy depends on data semantics.
• Monitor relentlessly. Queue depth, dropped counts, disconnect reasons, and Kafka consumer lag are your early warnings. Build dashboards before you need them.