Pub/sub vs point-to-point queues: lessons from production

Pub/sub vs point-to-point queues

Over the past seventy days, we've built systems that need to talk to each other without tight coupling. Today, I want to clarify one of the most consequential decisions you'll make when designing those communications: pub/sub vs point-to-point queues. These two messaging patterns solve fundamentally different problems, and mixing them up costs you consistency, scalability, or both.

Let me walk you through the real difference, show you when each shines, and explain how combining them creates one of the most powerful integration patterns in distributed systems.

Understanding the core difference

The distinction is simple but profound: pub/sub broadcasts to everyone; point-to-point routes to one consumer.

With pub/sub, you publish an event once and every subscriber who's listening gets a copy. Think of it as shouting across a room—everyone hears it. With point-to-point queues, you enqueue a job and exactly one worker picks it up and processes it. Think of it as handing a task to the first available person.

This fundamental difference cascades into everything else: semantics, durability guarantees, retry logic, and how you scale.

Pub/sub: Events and notifications

Pub/sub is the pattern you reach for when you need to broadcast facts—things that happened—to an open-ended set of interested parties. You don't know at publish time who will care. You might have zero subscribers today and five tomorrow. That's fine.

The classic example: a user signs up. That's a fact. It happened. Now, multiple systems want to react: send a welcome email, increment a signup counter, trigger onboarding workflows, log analytics. None of them should block the signup API. None of them should care that the others exist.

You publish a UserSignedUp event to a topic. Every subscriber to that topic receives it, independently and in parallel. If an email service is slow, it doesn't block analytics. If analytics crashes and restarts, it can replay the event from a retained log. Each subscriber is loosely coupled.

Kafka topics are the gold standard for pub/sub at scale. Topics retain messages for a configurable window (or indefinitely). New subscribers can start from the beginning, the end, or anywhere in between. Multiple consumer groups can process the same topic for different purposes.

// Kafka producer: publish once, many subscribers get it
const event = {
  eventType: 'UserSignedUp',
  userId: '12345',
  email: 'user@example.com',
  timestamp: new Date().toISOString()
};

await producer.send({
  topic: 'user-events',
  messages: [{ value: JSON.stringify(event) }]
});

// Email subscriber
consumer1.subscribe({ topic: 'user-events' });
consumer1.run({
  eachMessage: async ({ message }) => {
    const event = JSON.parse(message.value);
    if (event.eventType === 'UserSignedUp') {
      await sendWelcomeEmail(event.email);
    }
  }
});

// Analytics subscriber (independent, gets the same message)
consumer2.subscribe({ topic: 'user-events' });
consumer2.run({
  eachMessage: async ({ message }) => {
    const event = JSON.parse(message.value);
    if (event.eventType === 'UserSignedUp') {
      await incrementSignupCounter(event.userId);
    }
  }
});

AWS SNS (Simple Notification Service) is another popular pub/sub tool, especially in AWS-native stacks. It's simpler than Kafka, with less operational overhead, but fewer retention guarantees. Messages are typically discarded after a short period if not delivered.

Use pub/sub when:

• You're publishing immutable facts (events).
• Multiple, independent systems need to react.
• Subscribers may be added or removed without publisher changes.
• You benefit from replaying historical events.

Point-to-point queues: Jobs and tasks

Point-to-point queues are for work. A job arrives, one worker owns it, processes it to completion, and acknowledges it. The queue guarantees that exactly one worker will process that message. Once acknowledged, it's gone.

The classic example: a payment processing queue. You can't have two workers both charging the same credit card. You need exactly-once semantics. One worker picks up the job, processes the payment, and marks it done. If that worker crashes mid-process, the job goes back into the queue for another worker to retry.

RabbitMQ queues and AWS SQS are the go-to choices. They're simple, durable, and designed for work distribution. SQS is managed and scales effortlessly. RabbitMQ gives you more control and richer routing options.

// SQS: enqueue a payment job
await sqs.sendMessage({
  QueueUrl: paymentQueueUrl,
  MessageBody: JSON.stringify({
    jobId: 'pay-123',
    orderId: 'order-456',
    amount: 99.99,
    cardToken: 'tok_xyz'
  })
});

// Worker processes one job at a time
while (true) {
  const messages = await sqs.receiveMessage({
    QueueUrl: paymentQueueUrl,
    MaxNumberOfMessages: 1,
    WaitTimeSeconds: 20
  });

  if (!messages.Messages) continue;

  const message = messages.Messages[0];
  const job = JSON.parse(message.Body);

  try {
    // Process the job
    await processPayment(job);
    
    // Acknowledge (delete from queue)
    await sqs.deleteMessage({
      QueueUrl: paymentQueueUrl,
      ReceiptHandle: message.ReceiptHandle
    });
  } catch (error) {
    // Don't acknowledge; message goes back for retry
    console.error('Payment failed', error);
  }
}

Point-to-point queues offer strong delivery guarantees: at-least-once (the default) or exactly-once (with idempotent deduplication). They handle backpressure elegantly—if workers are slow, messages accumulate. You can autoscale workers based on queue depth.

Use point-to-point queues when:

• You're distributing work (jobs, tasks, commands).
• Exactly-once processing is critical.
• A single consumer should own the entire lifecycle.
• You need built-in retry and dead-letter handling.

Combining them: Fan-out pattern

Here's where it gets interesting. You don't have to choose one or the other globally. The fan-out pattern uses both together, each for its strength.

Publish an event to a pub/sub topic. That topic fans out to multiple point-to-point queues. Each queue is owned by a different service. This gives you the best of both worlds: loosely coupled event broadcast with point-to-point work guarantees.

Real-world scenario: an order is placed. You publish OrderPlaced to SNS. SNS fans it out to three SQS queues: one for the payment service, one for fulfillment, one for analytics. The payment service pulls from its queue and processes the payment with exactly-once semantics. Fulfillment pulls and picks items. Analytics pulls and logs. If fulfillment crashes during processing, the message stays in its queue. Payment's work is independent.

// Step 1: Publish to SNS topic
const sns = new AWS.SNS();
await sns.publish({
  TopicArn: 'arn:aws:sns:us-east-1:123456789:order-events',
  Message: JSON.stringify({
    eventType: 'OrderPlaced',
    orderId: 'order-789',
    userId: 'user-123',
    items: [{ sku: 'item-1', qty: 2 }],
    totalAmount: 49.99
  })
});

// Step 2: SNS subscribers route to separate SQS queues
// (configured in AWS console or CDK)
// - SNS routes to SQS payment queue
// - SNS routes to SQS fulfillment queue
// - SNS routes to SQS analytics queue

// Step 3: Each service pulls from its own queue
const paymentWorker = async () => {
  const sqs = new AWS.SQS();
  while (true) {
    const messages = await sqs.receiveMessage({
      QueueUrl: paymentQueueUrl,
      MaxNumberOfMessages: 10
    });

    for (const message of messages.Messages || []) {
      const order = JSON.parse(message.Body);
      try {
        await chargeCard(order);
        await sqs.deleteMessage({
          QueueUrl: paymentQueueUrl,
          ReceiptHandle: message.ReceiptHandle
        });
      } catch (err) {
        // Retry automatically after visibility timeout
      }
    }
  }
};

This pattern is powerful because:

• Publishers don't know about subscribers (loose coupling).
• Each subscriber has its own queue, so slowness in one doesn't affect others.
• Each queue can have its own concurrency, retry policy, and dead-letter handling.
• You can add a new subscriber by adding a new queue subscription without changing the publisher.

Choosing by intent

The decision really comes down to this: Are you sharing facts or assigning work?

Facts are immutable. They happened. Multiple parties care. Examples: user signed up, payment processed, order shipped, config changed. These go to pub/sub. The publisher doesn't care who listens or what they do.

Work is mutable and sequential. Someone has to do it. Exactly one owner at a time. Examples: send an email, charge a card, process a refund, generate a report. These go to queues. The publisher knows work needs to happen and enqueues it for the next available worker.

If you're not sure, ask: Does this need to be processed exactly once by one owner, or do multiple systems need to independently react?

Common mistakes

• Using pub/sub for work. Events fire and forget. If a subscriber misses it, there's no retry. Use queues for critical work.
• Using queues for events. You'll end up with multiple queues hardcoded to the publisher. Adding a new subscriber means changing the publisher. Use pub/sub for open-ended broadcast.
• Forgetting retention windows. SNS messages disappear after 15 minutes if not delivered. Kafka retains based on your config. If you need replays, pick accordingly.
• Mixing concerns. A message shouldn't be "here's an event, but also go do this job." Publish the event; let interested services enqueue their own work based on it.
• Ignoring ordering. Kafka partitions guarantee ordering within a partition. SQS does not. If order matters, plan accordingly.

Key takeaways

• Pub/sub broadcasts events to all subscribers independently; point-to-point queues route work to exactly one consumer.
• Pub/sub is for facts (what happened); queues are for work (what needs to be done).
• Kafka topics retain messages and allow replay; SNS is simpler but messages vanish quickly.
• SQS and RabbitMQ queues guarantee exactly-once delivery with retry semantics built in.
• Fan-out (SNS to multiple SQS queues) combines both patterns for loosely coupled, reliable systems.
• Choose based on intent: multiple independent reactions = pub/sub; single owner processes once = queue.
• Don't hardcode subscriber queues into publishers; use pub/sub to broadcast and let subscribers self-subscribe.

This distinction has saved me from countless architectural headaches. Next time you're designing a message flow, pause and ask: fact or work? Your system's consistency and maintainability depend on getting this right.