Scaling WebSockets horizontally: lessons from production

Scaling WebSockets horizontally

You've built a real-time chat feature or live notification system that works perfectly on a single server, but the moment you spin up a second instance behind a load balancer, messages stop flowing between users on different servers. Scaling WebSockets horizontally—distributing connections across multiple servers while maintaining a unified experience—is one of the trickiest challenges in distributed systems, and it requires rethinking how you broadcast state.

In this post, I'll walk through the core problem, the pub/sub patterns that solve it, and the load-balancing strategies that keep your architecture stable under real-world traffic.

The problem — Isolated server instances

When you run a single WebSocket server, every connected client holds an in-memory connection object. When client A sends a message, you iterate over all connected sockets and emit the data. Simple.

Now imagine two servers. Client A connects to Server 1, Client B connects to Server 2. When A sends a message, Server 1 has no reference to B's socket, so the message never arrives. Each server has its own isolated pool of connections.

This is especially painful because:

• Broadcasts fail silently. Your application logic looks correct—you're calling emit on the right events—but half your users never see the data.
• Presence tracking breaks. If you're tracking "who's online," Server 1 knows about its clients and Server 2 knows about its own. No global view.
• Room-based logic collapses. A "room" or "channel" concept only works within a single server instance.

I hit this wall early in my career when scaling a live trading dashboard from 1 server to 3. Traders on different instances couldn't see each other's activity. The root cause: I was broadcasting to local connections only, not across the entire cluster.

Pub/sub backplane — Broadcasting across instances

The solution is a pub/sub backplane: a shared message broker that sits between your servers. When any server needs to broadcast, it publishes to the backplane instead of only to local sockets. Every other server subscribes to the same channels and forwards messages to their connected clients.

I typically use Redis for this because it's fast, widely available, and Socket.IO (the most popular WebSocket library) has a first-class Redis adapter.

Redis adapter pattern

Here's a minimal NestJS example with Socket.IO and the Redis adapter:

import { Module } from '@nestjs/common';
import { WebSocketGateway, WebSocketServer, SubscribeMessage } from '@nestjs/websockets';
import { createAdapter } from '@socket.io/redis-adapter';
import { createClient } from 'redis';
import { Server } from 'socket.io';

const pubClient = createClient();
const subClient = pubClient.duplicate();

@WebSocketGateway({
  namespace: 'chat',
})
@Module({})
export class ChatGateway {
  @WebSocketServer() server: Server;

  async onModuleInit() {
    await Promise.all([pubClient.connect(), subClient.connect()]);

    this.server.adapter(
      createAdapter(pubClient, subClient),
    );
  }

  @SubscribeMessage('message')
  handleMessage(client: any, payload: { text: string; room: string }) {
    // Emit to room. Redis adapter forwards to all servers.
    this.server.to(payload.room).emit('message', {
      text: payload.text,
      userId: client.id,
      timestamp: new Date(),
    });
  }
}

What happens here:

1. Server 1 receives a message event from Client A.
2. The handler calls this.server.to(room).emit().
3. The Redis adapter intercepts this and publishes to Redis on the channel socket.io#room.
4. Server 2, 3, and any other instance subscribed to that channel receive the message.
5. Each server emits the data to its own connected clients in that room.

Why Redis? It's simple, proven, and low-latency. I've run this pattern with hundreds of concurrent connections. For truly massive scale (thousands of rooms, millions of connections), some teams move to Kafka, but Redis is the sweet spot for most applications.

Alternative: Kafka-based pub/sub

If you're already using Kafka for your event stream (order events, user activity, etc.), you can wire WebSocket broadcasts into those topics. This couples your real-time layer to your event architecture, which can be powerful—every trade, purchase, or status update automatically flows to subscribed clients.

// Kafka consumer listening to 'trades' topic
consumer.on('message', async (message) => {
  const trade = JSON.parse(message.value.toString());
  
  // Forward to all clients watching this symbol
  this.wss.to(`symbol:${trade.symbol}`).emit('tradeUpdate', trade);
});

// But remember: Kafka introduces latency and complexity.
// Use it only if you need the durability and ordering guarantees.

The tradeoff: Kafka is more reliable and persistent, but adds latency and operational overhead. I reach for Kafka when trades or financial events must never be lost; I use Redis when real-time speed matters more than durability.

Sticky sessions — Keeping fallback connections stable

WebSocket is the ideal protocol for long-lived bidirectional communication. But not all clients support it. Some proxies, firewalls, or older browsers fall back to long polling, where the client repeatedly opens short HTTP requests to check for new data.

Here's the problem: a load balancer using round-robin routing might send Request 1 to Server A and Request 2 to Server B. If Server B doesn't know about this client's session, it returns no data. The client has to wait for the next poll cycle and hope it lands on the right server.

Sticky sessions (session affinity) pin a client to a single server for the duration of the connection. Once Client C hits Server 1, all subsequent requests go to Server 1 until the connection closes.

Implementing sticky sessions

Most load balancers support this natively. In AWS ALB, enable target group stickiness:

// AWS CDK example
const targetGroup = new elbv2.ApplicationTargetGroup(this, 'WebSocketTargets', {
  port: 3000,
  vpc,
  protocol: elbv2.ApplicationProtocol.HTTP,
  stickynessType: elbv2.TargetGroupStickiness.LB_COOKIE,
  stickinessCookieDuration: cdk.Duration.days(1),
});

In Nginx:

// nginx.conf snippet
upstream ws_backend {
    least_conn;  // See next section
    server 10.0.1.10:3000;
    server 10.0.1.11:3000;
    server 10.0.1.12:3000;
}

server {
    listen 80;
    server_name example.com;

    location /socket.io {
        proxy_pass http://ws_backend;
        proxy_set_header Connection "Upgrade";
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        
        // Sticky session via cookie
        proxy_cookie_flags ~ secure httponly;
    }
}

Important: sticky sessions are a fallback for clients that can't maintain true WebSocket. Modern browsers and mobile apps use native WebSocket, which persists a TCP connection. Sticky sessions mainly help during the long-polling negotiation phase.

I've seen teams overthink this. If your clients are modern (99% of cases), invest your effort elsewhere. But if you're supporting older corporate environments, sticky sessions are non-negotiable.

Connection-aware load balancing — Least-connections algorithm

Round-robin load balancing assigns incoming requests equally across servers. It's fair for stateless HTTP, but terrible for long-lived WebSocket.

Imagine three servers and a stream of new WebSocket connections. Round-robin assigns them: Server A (1 connection), Server B (1 connection), Server C (1 connection), Server A (2 connections), Server B (2 connections), etc. Perfect distribution so far.

But if Server A then receives 100 simultaneous connections from a single client (a real-time trading floor connecting all workstations), round-robin is oblivious. It keeps sending new connections to Server A until the assignment cycle completes, overloading it while Servers B and C sit idle.

Least-connections routing tracks active connections per server and routes new requests to the server with the fewest active connections. This spreads load more intelligently.

Configuring least-connections

In Nginx:

upstream ws_backend {
    least_conn;  // Use least-connections instead of round-robin
    server 10.0.1.10:3000 max_fails=3 fail_timeout=30s;
    server 10.0.1.11:3000 max_fails=3 fail_timeout=30s;
    server 10.0.1.12:3000 max_fails=3 fail_timeout=30s;
}

In HAProxy:

backend websocket_servers
    balance leastconn  // Least connections algorithm
    server srv1 10.0.1.10:3000 check
    server srv2 10.0.1.11:3000 check
    server srv3 10.0.1.12:3000 check
    option forwardfor
    default-server inter 10s fall 3

Real-world impact: I once debugged why a single server was consuming 80% CPU while two peers were at 15%. Round-robin had been routing new connections to that server in bulk. Switching to least-connections evened the load within minutes.

Combine least-connections with pub/sub, and you have a robust horizontally scaled WebSocket layer. Clients are distributed by connection count, messages flow across the cluster via Redis, and fallback connections stay sticky.

Common mistakes

• Forgetting the backplane entirely. Running multiple servers without a pub/sub adapter is the most common mistake. You get partial functionality (some users see messages, others don't), making it hell to debug.
• Over-engineering with Kafka. Kafka is powerful but slow. I've seen teams add 200–500ms of latency trying to make trades "event-sourced." Redis pub/sub is <5ms and sufficient for 99% of real-time use cases.
• Assuming WebSocket works everywhere. Old proxies, restrictive corporate networks, and certain mobile carriers don't support WebSocket. Always test with fallback transport (long polling, SSE) and ensure your load balancer handles it correctly.
• Not monitoring connection distribution. Set up alerts on connection count per server. A sudden spike on one instance often signals a load-balancer misconfiguration or a bug in your auto-scaling logic.
• Sticky sessions as a substitute for backplane. Some teams think "if I stick connections to a server, I don't need pub/sub." Wrong. Sticky sessions only help clients that can't maintain long-lived connections. Your broadcast logic still needs a backplane.

Key takeaways

• A single server can handle WebSocket connections, but scaling horizontally requires a pub/sub backplane (Redis or Kafka) so messages broadcast across all instances.
• Use the Redis Socket.IO adapter for most applications—it's simple, fast (<5ms), and production-proven. Reserve Kafka for scenarios where durability and ordering are critical.
• Sticky sessions are essential only if you support long-polling fallback. Modern clients use native WebSocket, so this is often a non-issue.
• Replace round-robin with least-connections routing on your load balancer. It spreads long-lived connections more fairly and prevents hotspots.
• Monitor connection distribution per server and test fallback transports in your staging environment. WebSocket scaling breaks silently—lack of observability makes it hard to catch.