The snapshot-plus-delta pattern: lessons from production

The snapshot-plus-delta pattern

When you have thousands of clients connected via WebSockets, sending the entire state to every client every time something changes becomes prohibitively expensive. The snapshot-plus-delta pattern solves this by sending a complete state snapshot once on subscription, then only broadcasting the changes that follow. This approach has saved me countless infrastructure costs across e-commerce platforms, trading systems, and real-time dashboards—and it's simpler to implement than you'd think.

Why full state updates don't scale

Early in my career, I built a product listing service that pushed the entire inventory state to connected clients whenever any item changed. With a few hundred concurrent users, the network traffic and CPU usage became unbearable. Each product update triggered a broadcast of 5–10 MB of data to every client, even though only one field changed. The problem compounds with Kafka: if you're using Kafka as your event source, every topic subscriber would receive and process that full payload, multiplying the load.

The root issue is that bandwidth and compute are finite. At scale, you don't push redundancy; you push only what changed. This is where the snapshot-plus-delta pattern becomes essential.

Send a full snapshot on subscribe

The pattern begins the moment a client connects. Before any deltas (changes) are sent, you transmit the complete current state as a single message. This gives the client a known baseline from which all future updates can be applied incrementally.

Here's a practical example. Imagine a ride-hailing system where clients need to see available drivers in real time:

// Server: When a client subscribes to the "drivers" channel
websocket.on('subscribe', async (message) => {
  const { channel, clientId } = message;

  if (channel === 'drivers') {
    // Fetch the full current state
    const allDrivers = await db.drivers.find({ status: 'active' });

    // Send snapshot with a snapshot version number
    const snapshot = {
      type: 'snapshot',
      version: 1001,
      data: allDrivers,
      timestamp: Date.now(),
    };

    websocket.send(JSON.stringify(snapshot));
    console.log(`Sent snapshot v${snapshot.version} to ${clientId}`);
  }
});

The snapshot includes a version number (I'll explain why shortly). It's also typically a separate message type so clients know to replace their local cache entirely, rather than merge it. The snapshot is sent before any delta subscriptions are activated, ensuring clients always have a clean state to start from.

Then send only changes

Once the client has the snapshot, subsequent broadcasts contain only the delta—the minimal information about what changed. A delta might be a new driver's location, a status update, or a price change, not the entire dataset.

Continuing the ride-hailing example:

// Kafka consumer listening for driver events
kafkaConsumer.on('message', async (kafkaMessage) => {
  const event = JSON.parse(kafkaMessage.value.toString());

  // This could be a driver location update
  if (event.type === 'driver_location_updated') {
    const delta = {
      type: 'delta',
      sequenceNumber: ++globalSequence,
      change: {
        entityId: event.driverId,
        field: 'location',
        value: event.newLocation,
      },
      timestamp: Date.now(),
    };

    // Broadcast to all subscribed clients
    broadcastToSubscribers('drivers', JSON.stringify(delta));
  }
});

Notice the sequenceNumber. This is critical. Each delta increments a sequence counter, allowing clients to detect if they've missed any updates. The delta is minimal: only the entity ID, field name, and new value. Compare this to resending the entire driver object or all drivers—the bandwidth reduction is massive.

In production systems I've worked on, a single delta message is typically 200–500 bytes, while a full snapshot can be 2–5 MB. At 1,000 concurrent clients, the difference is immediate and obvious in your AWS bill.

Sequence numbers detect gaps

The client side is equally important. When receiving deltas, your client must track the last sequence number it successfully applied. If the next delta has a gap—say, you received sequence 105 then 108, missing 106 and 107—you know some updates were lost.

Here's how a client tracks this:

// Client: WebSocket message handler
let lastSequenceNumber = 0;
let driverCache = {};

websocket.onmessage = (event) => {
  const message = JSON.parse(event.data);

  if (message.type === 'snapshot') {
    // Reset state on snapshot
    driverCache = {};
    message.data.forEach((driver) => {
      driverCache[driver.id] = driver;
    });
    lastSequenceNumber = message.version;
    console.log(`Initialized cache with ${message.data.length} drivers`);
  }

  if (message.type === 'delta') {
    const gap = message.sequenceNumber - lastSequenceNumber - 1;
    
    if (gap > 0) {
      console.warn(`Detected gap of ${gap} messages. Requesting new snapshot.`);
      websocket.send(JSON.stringify({ action: 'request_snapshot', channel: 'drivers' }));
      return; // Don't apply this delta until gap is resolved
    }

    // Apply the delta
    const { entityId, field, value } = message.change;
    if (driverCache[entityId]) {
      driverCache[entityId][field] = value;
    }
    lastSequenceNumber = message.sequenceNumber;
  }
};

The client checks message.sequenceNumber - lastSequenceNumber - 1. If this is greater than zero, there's a gap. The client records which sequence numbers are missing and can request a replay or a fresh snapshot. Without sequence numbers, a client would silently become out of sync, showing stale data to the user.

Request a new snapshot when gaps appear

When a gap is detected, the client should request a fresh snapshot rather than trying to replay individual deltas. Why? Network conditions are unpredictable. A client might miss messages due to temporary disconnection, browser tab backgrounding, or a lossy connection. Replaying missing deltas adds complexity; requesting a new snapshot is simpler and more reliable.

On the server side, handle these requests by sending an updated snapshot:

// Server: Handle snapshot request
websocket.on('message', async (rawMessage) => {
  const message = JSON.parse(rawMessage);

  if (message.action === 'request_snapshot') {
    const { channel } = message;
    
    // Fetch fresh state
    const currentState = await db.getChannelState(channel);
    
    const newSnapshot = {
      type: 'snapshot',
      version: ++snapshotVersionCounter,
      data: currentState,
      timestamp: Date.now(),
    };

    websocket.send(JSON.stringify(newSnapshot));
    console.log(`Sent fresh snapshot v${newSnapshot.version} to client after gap detection`);
  }
});

The server increments a global snapshot version number. This allows you to correlate snapshots across clients and debug issues where certain clients are out of sync. Also consider implementing exponential backoff on the client—if snapshot requests keep failing, pause for a bit before retrying.

Common mistakes

• Not including sequence numbers in deltas. Without them, you can't detect gaps. Some teams try to use timestamps instead; don't. Timestamps are not monotonically increasing across distributed Kafka brokers. Always use a sequence counter.
• Applying deltas without validating the snapshot first. If the client hasn't received a snapshot, applying a delta to an empty cache corrupts the state. Always send the snapshot before opening the delta stream.
• Forgetting that snapshot versions and delta sequences are separate. A snapshot has a version; deltas have sequence numbers. Don't mix them. Your client needs to track both.
• Sending deltas before the snapshot has been acknowledged. Use a separate subscription acknowledgement message. Have the client confirm it's received and cached the snapshot before the server starts broadcasting deltas to that client.
• Not resetting sequence numbers on new snapshots. When you send a fresh snapshot, increment its version but reset the delta sequence counter to match. This prevents confusion about which sequence number comes next.
• Over-buffering deltas on the server. If a client disconnects and reconnects, don't try to replay 10,000 deltas from a buffer. Send a new snapshot instead. Buffers add complexity and memory overhead.

Key takeaways

• The snapshot-plus-delta pattern reduces bandwidth by orders of magnitude compared to full-state broadcasts. A single snapshot up front, then only changes after, scales to thousands of concurrent clients.
• Sequence numbers in deltas are non-negotiable. They let clients detect gaps and know when to request a fresh snapshot. Without them, clients silently diverge from server state.
• Gaps happen. Network hiccups, temporary disconnections, and packet loss are normal. Design for them by allowing clients to request a new snapshot whenever a gap is detected.
• Keep snapshots and deltas as separate message types. This makes client-side logic clearer and prevents accidentally merging a delta into an uninitialized cache.
• In systems where Kafka feeds your WebSocket broadcasts, the delta structure should be derived directly from the Kafka event. Don't transform events into deltas; let the Kafka consumer publish deltas directly.
• Test gap scenarios explicitly. Disconnect clients mid-stream, delay delta delivery, and verify that clients recover gracefully by requesting new snapshots.