Chat system design basics: practical tips and trade-offs

Chat system design is a perennial interview favourite because it forces you to think about ordering, consistency, and real-time delivery all at once. It's also a problem you'll face in production—whether you're building an in-app messenger, a customer support system, or a notification feed. This article walks through chat system design basics: practical tips and trade-offs, grounded in what actually works at scale.

Message Storage — Persist First, Then Deliver

The golden rule: never rely on delivery as your source of truth. Store every message to a persistent database before you even think about sending it to clients.

Here's why this matters. A user sends a message. Your system publishes it to a Kafka topic for real-time subscribers. The subscriber crashes mid-delivery. Now that message is gone for offline users—and you have no record of what happened. By the time the offline user comes back online, there's no way to replay the conversation accurately.

The pattern that works:

1. User sends message via REST or WebSocket.
2. Write to your primary database (PostgreSQL, DynamoDB, whatever you use).
3. Publish to Kafka for real-time subscribers.
4. Return success to the client only after step 2 completes.

In practice, your messages table might look like this:

-- PostgreSQL schema
CREATE TABLE messages (
  id BIGSERIAL PRIMARY KEY,
  conversation_id BIGINT NOT NULL,
  sender_id BIGINT NOT NULL,
  body TEXT NOT NULL,
  sequence_number BIGINT NOT NULL,
  created_at TIMESTAMP DEFAULT NOW(),
  UNIQUE(conversation_id, sequence_number)
);

CREATE INDEX idx_conv_sequence ON messages(conversation_id, sequence_number);

Notice the sequence_number column. More on that in the ordering section. The key here is that every write is durable before you signal success to the client. Your Kafka publish can fail, and the message is still safe.

Trade-off: You add latency. A client won't see "message sent" until the database write completes. For most chat systems, 50–200ms of extra latency is acceptable. If you need sub-50ms, you're in a different world (high-frequency trading, real-time collaboration) and need different patterns.

Delivery — WebSocket for Online Users, Push for Offline Users

Real-time delivery splits into two channels: those connected right now (WebSocket), and those who aren't (push notifications or email).

Online Delivery via WebSocket

When a user is connected, you can push messages directly. This is where Kafka shines. Publish the message to a topic named after the conversation, e.g., chat-conversation-12345. Your WebSocket server subscribes to that topic and broadcasts to all connected clients in that conversation.

// Pseudo-code: WebSocket handler on a NestJS/Express server
const kafka = new Kafka({ /* config */ });
const consumer = kafka.consumer({ groupId: 'websocket-group' });

io.on('connection', (socket) => {
  socket.on('join-conversation', async (conversationId) => {
    // Subscribe to Kafka topic for this conversation
    const topic = `chat-conversation-${conversationId}`;
    await consumer.subscribe({ topic });
    
    consumer.run({
      eachMessage: async ({ message }) => {
        const parsedMsg = JSON.parse(message.value);
        socket.emit('message', parsedMsg);
      }
    });
  });
});

This pattern lets you scale WebSocket servers horizontally. Any server can handle any client, because the source of truth is Kafka. If a client reconnects to a different server, it just subscribes to the same Kafka topic and catches up.

Offline Delivery

When a user goes offline, WebSocket delivery stops. But they should still know they have new messages. This is where push notifications or email come in. Store an unread message counter or queue, then trigger a push notification if the user has opted in and isn't connected.

A simple approach: after writing a message to the database, check if the recipient is online (e.g., via a Redis cache of active connections). If not, queue a push notification task (use a job queue like Bull, Celery, or even Kafka).

// After message is stored in DB
async function sendMessage(conversationId, senderId, body) {
  const msg = await db.messages.create({
    conversation_id: conversationId,
    sender_id: senderId,
    body,
    sequence_number: await getNextSequence(conversationId)
  });

  // Publish to Kafka for online subscribers
  await producer.send({
    topic: `chat-conversation-${conversationId}`,
    messages: [{ value: JSON.stringify(msg) }]
  });

  // Check if recipient is online
  const recipientId = await getConversationRecipient(conversationId, senderId);
  const isOnline = await redis.get(`user:${recipientId}:online`);
  
  if (!isOnline) {
    // Queue a push notification
    await pushQueue.add({
      userId: recipientId,
      conversationId,
      type: 'new-message'
    });
  }
}

Trade-off: You're now managing two separate delivery channels. Push notifications are slower and less reliable than WebSocket (dependent on mobile OS, network, user settings). But they're essential for keeping users engaged when they're not actively using your app.

Ordering — Per-Conversation Sequence Numbers

Messages must appear in the same order on every client. Clock-based ordering (timestamps) breaks down in distributed systems—clocks drift, and you get race conditions. Sequence numbers solve this.

Assign a monotonically increasing sequence number to each message within a conversation. When a client reconnects, it fetches all messages with sequence number > its last seen number. This guarantees no gaps and no duplicates.

-- When fetching message history
SELECT * FROM messages
WHERE conversation_id = ?
  AND sequence_number > ?
ORDER BY sequence_number ASC
LIMIT 50;

Generate the sequence number atomically when you write to the database. In PostgreSQL, a simple approach is to use a function:

CREATE OR REPLACE FUNCTION insert_message(
  p_conversation_id BIGINT,
  p_sender_id BIGINT,
  p_body TEXT
) RETURNS messages AS $$
DECLARE
  v_sequence BIGINT;
  v_message messages;
BEGIN
  -- Get the next sequence number for this conversation
  SELECT COALESCE(MAX(sequence_number), 0) + 1
  INTO v_sequence
  FROM messages
  WHERE conversation_id = p_conversation_id;

  -- Insert with the sequence number
  INSERT INTO messages (conversation_id, sender_id, body, sequence_number)
  VALUES (p_conversation_id, p_sender_id, p_body, v_sequence)
  RETURNING * INTO v_message;

  RETURN v_message;
END;
$$ LANGUAGE plpgsql;

Trade-off: This approach requires a database round-trip for every write. If you have high message volume, you might want to batch inserts or use a distributed counter (e.g., Redis or a separate sequence service). But for most chat systems, a single database call per message is acceptable.

Read Receipts — Track Last Read Message Per User Per Conversation

Users want to know if their message was seen. This is a simple state-tracking problem: for each user in each conversation, store the highest sequence number they've read.

Create a separate table:

CREATE TABLE conversation_reads (
  id BIGSERIAL PRIMARY KEY,
  conversation_id BIGINT NOT NULL,
  user_id BIGINT NOT NULL,
  last_read_sequence BIGINT DEFAULT 0,
  updated_at TIMESTAMP DEFAULT NOW(),
  UNIQUE(conversation_id, user_id)
);

When a user views a message, update this table. You don't need to do this for every single message—batch updates every 500ms or when the user scrolls to a certain threshold.

// Client: batch read receipts
let pendingReadUpdate: NodeJS.Timeout;

socket.on('message-view', (messageSequence) => {
  clearTimeout(pendingReadUpdate);
  
  pendingReadUpdate = setTimeout(() => {
    socket.emit('read-receipt', {
      conversationId,
      lastReadSequence: messageSequence
    });
  }, 500);
});

// Server: update and broadcast
socket.on('read-receipt', async (data) => {
  await db.conversationReads.upsert({
    conversation_id: data.conversationId,
    user_id: socket.userId,
    last_read_sequence: data.lastReadSequence
  }, {
    conflictFields: ['conversation_id', 'user_id']
  });

  // Broadcast to other participants
  io.to(`conv-${data.conversationId}`).emit('user-read', {
    userId: socket.userId,
    lastReadSequence: data.lastReadSequence
  });
});

When fetching message history, join this table to show which messages have been read by which users. This is especially useful in group chats.

Trade-off: Read receipt updates are asynchronous and lower priority than the messages themselves. A user might see a message but the "read" status doesn't sync immediately. That's fine—a few seconds of delay is usually unnoticed. If you need sub-second read receipt accuracy, you're adding complexity for minimal user benefit.

Common Mistakes

• Skipping the database. Tempting to use only Kafka or only Redis. It breaks. Always persist first.
• Using timestamps for ordering. Clocks aren't ordered. Sequence numbers are. Use them.
• Ignoring offline users. It's easy to build a WebSocket-only system that works until someone closes the app. Push notifications (or at least a notification center) are essential.
• Over-engineering read receipts. Tracking every keystroke or every millisecond adds load for little user value. Batch and debounce aggressively.
• Not testing reconnection scenarios. Close your browser DevTools and reconnect. Does message history sync correctly? Do you get duplicates? This is where bugs hide.

Key Takeaways

• Persist messages to a database before signalling success. This is your source of truth, not Kafka or WebSocket.
• Use Kafka to deliver to online users (via WebSocket servers) and a separate job queue for offline push notifications.
• Assign sequence numbers per conversation to guarantee ordering. Timestamps don't work in distributed systems.
• Track read receipts with a simple per-user, per-conversation state table. Batch updates to reduce load.
• Test reconnection and offline scenarios aggressively. That's where real chat systems fail.