Load balancers and WebSockets: best practices explained

The WebSocket proxy problem we all face

Most WebSocket outages aren't caused by your application code—they're caused by misconfigured load balancers and reverse proxies sitting between clients and servers. We've seen this pattern repeatedly across trading platforms, ride-hailing services, and real-time SaaS products: connections drop mysteriously, clients reconnect in waves, and support tickets flood in. Load balancers and WebSockets: best practices explained covers the configuration decisions that separate stable deployments from ones that hemorrhage connections.

The gap exists because HTTP and WebSocket require different proxy behavior. Your load balancer works fine for stateless REST APIs—it can drop idle connections, route requests freely, and use connection pooling. WebSocket breaks all those assumptions. Once a connection upgrades, it becomes stateful, long-lived, and requires careful handling. Getting this right prevents the majority of production incidents we've debugged.

Allow the Upgrade and Connection headers

The WebSocket handshake starts as HTTP. A client sends a GET request with two critical headers: Connection: Upgrade and Upgrade: websocket. These tell the server and every proxy in between that the connection should switch protocols. If your load balancer strips, blocks, or modifies these headers, the upgrade fails silently and clients either hang or fall back to polling.

Many proxies are configured to normalize or filter headers for security. NGINX, HAProxy, and cloud load balancers often have default rules that interfere. Here's what you need to ensure reaches your WebSocket server:

// Client sends (browser WebSocket API handles this)
GET /socket.io/?EIO=4&transport=websocket HTTP/1.1
Host: api.example.com
Connection: Upgrade
Upgrade: websocket
Sec-WebSocket-Key: [random base64]
Sec-WebSocket-Version: 13
Origin: https://app.example.com

For NGINX, the configuration is:

upstream websocket_backend {
  server ws1.internal:3000;
  server ws2.internal:3000;
}

server {
  listen 443 ssl http2;
  server_name api.example.com;

  location /socket.io/ {
    proxy_pass http://websocket_backend;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
  }
}

The key line is proxy_set_header Upgrade $http_upgrade;—this preserves the client's Upgrade header exactly. Without it, NGINX defaults to ignoring the header, and the connection never upgrades from HTTP.

For HAProxy, use http-server-close is not compatible with WebSocket. You need http-keep-alive or remove http-server-close entirely for WebSocket routes:

backend websocket_pool
  balance roundrobin
  option http-keep-alive
  server ws1 ws1.internal:3000 check
  server ws2 ws2.internal:3000 check

AWS ALB and NLB handle Upgrade headers correctly by default, but verify your security group rules don't block the port or any intermediate WAF rules don't filter the connection. We've seen WAF rules reject requests with custom headers, accidentally killing WebSocket handshakes.

Raise idle timeouts above heartbeat intervals

After the handshake completes, the WebSocket connection is open. Both client and server can send frames at any time. If traffic is sparse—a notification app waiting for push events, for example—the connection might sit idle for 30 seconds or more between messages.

Load balancers have idle timeouts, typically 60 seconds by default. If no data flows in either direction for that period, the proxy assumes the connection is dead and closes it. The client and server don't know this happened. The next time the client sends a message, it hangs because the proxy's connection is gone. The server never sees the request.

To prevent this, implement a heartbeat on the server side. Every 15 to 25 seconds, send a ping frame. The client responds with a pong. This keeps the connection "active" from the proxy's perspective. Then set the load balancer's idle timeout well above your heartbeat interval—ideally 2–3 times higher.

In a NestJS WebSocket server with Socket.IO:

@WebSocketGateway({
  cors: { origin: process.env.CLIENT_URL },
  pingInterval: 20000,  // Send ping every 20 seconds
  pingTimeout: 5000,    // Wait 5 seconds for pong before closing
})
export class EventsGateway implements OnGatewayConnection {
  @SubscribeMessage('message')
  handleMessage(client: Socket, data: any): void {
    client.broadcast.emit('message', data);
  }
}

On the NGINX side, ensure your proxy timeout is higher:

location /socket.io/ {
  proxy_pass http://websocket_backend;
  proxy_read_timeout 60s;      // Wait 60s before closing idle socket
  proxy_send_timeout 60s;
  proxy_http_version 1.1;
  proxy_set_header Upgrade $http_upgrade;
  proxy_set_header Connection "upgrade";
}

If your heartbeat is every 20 seconds, a 60-second timeout is safe. Never set it lower than your heartbeat interval. For cloud load balancers, check the docs carefully. AWS ALB defaults to 60 seconds; adjust it in your target group settings. Azure Load Balancer has a 4-minute default, which is usually fine. GCP Cloud Load Balancer has a 30-second default—raise it if your heartbeat is longer.

Enable sticky sessions when needed

Once a WebSocket connection is open, all subsequent frames must route to the same backend server. A client connected to server A cannot suddenly send a frame to server B—there's no connection there, and the server B has no memory of the client's session state.

Most load balancers default to round-robin across backends. This works fine for stateless HTTP requests. For WebSocket, round-robin breaks connections immediately. The first request (the handshake) routes to server A. The next frame routes to server B. Server B doesn't recognize the client and closes the connection.

Sticky sessions (also called session affinity) ensure that once a client connects to a backend, all future requests from that client go to the same server. The load balancer uses a cookie or IP hash to track this.

NGINX sticky sessions via IP hash:

upstream websocket_backend {
  ip_hash;  // Route based on client IP
  server ws1.internal:3000;
  server ws2.internal:3000;
  server ws3.internal:3000;
}

server {
  listen 443 ssl http2;
  location /socket.io/ {
    proxy_pass http://websocket_backend;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
  }
}

IP hash works if your clients have stable IPs (desktop users behind corporate NAT). It breaks if clients roam between networks or share NAT (mobile apps, public WiFi). A better approach is cookie-based affinity:

upstream websocket_backend {
  server ws1.internal:3000;
  server ws2.internal:3000;
  server ws3.internal:3000;
}

map $cookie_route $backend_pool {
  default "websocket_backend";
}

server {
  listen 443 ssl http2;
  location /socket.io/ {
    set $backend_route $cookie_route;
    if ($backend_route = "") {
      set $backend_route "ws1.internal:3000";
    }
    proxy_pass http://$backend_route;
    add_header Set-Cookie "route=$backend_route; Path=/; Max-Age=3600; Secure; HttpOnly" always;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
  }
}

AWS ALB and NLB support target stickiness out of the box. In your target group settings, enable stickiness with a 1-day duration. GCP Cloud Load Balancer uses client affinity based on IP and port. Azure App Gateway supports cookie-based affinity.

One gotcha: if you're running multiple WebSocket servers and want true session failover (if server A dies, clients reconnect to server B), sticky sessions alone won't help. You need shared state in Redis or a similar store so that any server can resume a client's session. This is more complex but necessary for high-availability ride-hailing or trading systems.

Drain connections during deploys

When you deploy a new version, you need to shut down the old server. If you kill it abruptly while clients are connected, they'll see an ungraceful disconnect and must reconnect. In a trading platform, this might trigger order cancellations. In ride-hailing, it might drop navigation updates.

Connection draining tells the load balancer: "Stop sending new connections to this server, but let existing ones finish." The server waits for clients to disconnect naturally (seconds to minutes), then shuts down. New clients route to updated servers immediately.

In NGINX, this is manual—you'd remove the server from the upstream block, reload the config, wait, then restart the service. For cloud load balancers, it's built-in. AWS calls it "connection draining," GCP calls it "connection drain timeout," Azure calls it "drain timeout."

Set the drain timeout to match your expected session lifetime. A notification app might drain in 30 seconds. A trading platform might need 5 minutes. The longer the timeout, the slower your deploy. The shorter the timeout, the more abrupt disconnects your clients face.

Better: implement graceful shutdown in your application. When the server receives a shutdown signal (SIGTERM), it sends a close frame to all connected clients with a code indicating the server is closing. Clients can reconnect immediately to a new server.

// In a NestJS gateway
async onApplicationShutdown(signal?: string) {
  console.log(`Received signal: ${signal}`);
  
  // Notify all connected clients
  for (const client of this.server.sockets.sockets.values()) {
    client.emit('server:shutting_down');
    client.disconnect(true);
  }
  
  // Wait for drains to complete
  await new Promise(resolve => setTimeout(resolve, 5000));
}

Pair this with a load balancer drain timeout of 10–15 seconds. The server closes connections gracefully; the load balancer ensures no orphaned connections remain.

Common mistakes

• Forgetting to set proxy_http_version 1.1: If you leave it at 1.0, the Upgrade header doesn't work. HTTP/1.1 is required for WebSocket.
• Setting idle timeouts too low: A 30-second timeout with a 20-second heartbeat is cutting it close. Use at least 60 seconds to account for jitter and network delays.
• Disabling sticky sessions for "load balancing": Round-robin looks better on dashboards (each server gets equal requests), but it breaks WebSocket. Use sticky sessions or lose connections.
• Not testing failover: Kill a backend server and watch what happens to connected clients. If they see ungraceful closes, improve your shutdown logic or increase drain timeouts.
• Proxy buffering: Some proxies buffer WebSocket frames to batch them. Disable buffering: proxy_buffering off; in NGINX. Buffering adds latency and can break frame ordering.

Key takeaways

• Preserve the Connection: Upgrade and Upgrade: websocket headers in your proxy configuration—they're non-negotiable for the handshake.
• Implement a heartbeat every 15–25 seconds and set load balancer idle timeouts to 2–3 times higher to prevent premature connection drops.
• Use sticky sessions (IP hash or cookie-based) to ensure all frames from a client route to the same backend server.
• Enable connection draining during deploys and implement graceful shutdown in your application to avoid abrupt disconnects.
• Test your proxy configuration with actual WebSocket clients under load before deploying to production.