When not to use Kafka for Kafka & WebSockets developers

After 47 days of building with Kafka and WebSockets, I've learned that the most important question isn't how to use these tools—it's whether you should use them at all. When not to use Kafka is just as critical as knowing when it shines, because I've watched teams spend months on infrastructure that could've been solved with a simple database queue and a cron job.

The Kafka seduction

Kafka is genuinely powerful. Distributed, durable, scalable, replay-capable. It looks like the answer to every async problem. But that appeal is exactly the trap. I've inherited systems where engineers added Kafka on day one "for when we scale," then never scaled. The operational burden killed productivity before throughput ever did.

Let me walk through four scenarios where I'd skip Kafka entirely and why.

Low-volume background jobs

Imagine you're running an e-commerce platform and need to send a confirmation email after each order. Maybe 50 orders a day. You could spin up Kafka, create an orders.created topic, attach a consumer service, configure partitions, set up monitoring. Or you could use a job queue.

I've handled this with Bull (Redis-backed queue), pg-boss (PostgreSQL-backed), or even a simple database table with a cron worker. Here's why they win:

• One less operational dependency
• Jobs are queryable, pausable, retryable from your database
• Dead-letter handling is simpler—just another table column
• Scaling is linear with database replication you already have

A concrete example: at a travel SaaS I worked on, we send booking confirmations, invoice PDFs, and reminder emails. Daily volume: 200–300 emails. We use a PostgreSQL table with a status column (pending, sent, failed) and a Node.js worker running every 5 seconds.

// migrations/001_create_email_jobs.sql
CREATE TABLE email_jobs (
  id SERIAL PRIMARY KEY,
  user_id INT NOT NULL,
  type VARCHAR(50) NOT NULL,
  data JSONB NOT NULL,
  status VARCHAR(20) DEFAULT 'pending',
  created_at TIMESTAMP DEFAULT NOW(),
  attempted_at TIMESTAMP,
  error_message TEXT
);

CREATE INDEX idx_pending_jobs ON email_jobs(status, created_at) 
WHERE status = 'pending';

// worker.ts
async function processEmailJobs() {
  const job = await db.query(
    `SELECT * FROM email_jobs 
     WHERE status = 'pending' 
     ORDER BY created_at ASC 
     LIMIT 1 FOR UPDATE SKIP LOCKED`
  );

  if (!job.rows.length) return;

  const emailJob = job.rows[0];
  try {
    await sendEmail(emailJob.user_id, emailJob.type, emailJob.data);
    await db.query(
      `UPDATE email_jobs SET status = 'sent', attempted_at = NOW() 
       WHERE id = $1`,
      [emailJob.id]
    );
  } catch (err) {
    await db.query(
      `UPDATE email_jobs 
       SET status = 'failed', error_message = $1, attempted_at = NOW() 
       WHERE id = $2`,
      [err.message, emailJob.id]
    );
  }
}

// Run every 5 seconds
setInterval(processEmailJobs, 5000);

This solution has delivered millions of emails. We see failures immediately in the UI, operators can retry single jobs, and we've never needed to tune Kafka.

A single service with no consumers

Sometimes people ask: "Should we publish events internally?" The answer is almost always no.

I see this at CRM platforms. Your backend creates a contact, and internally you want to sync it to Elasticsearch for search, trigger a webhook to a third-party, and log an audit event. You think: "Let's publish a contact.created event to Kafka, and three services consume it."

Here's the reality: there's only one service—yours. Those are internal features, not distributed systems.

• In-process event bus (EventEmitter in Node.js, IMediator in .NET) handles it perfectly
• No network latency, no serialization overhead, no deployment coordination
• Failures are synchronous and predictable—easier to debug
• If you truly need async, you already have the job queue from the previous section

Here's what I actually do:

// event-bus.ts
export class EventBus {
  private handlers = new Map>();

  subscribe(event: string, handler: Function) {
    if (!this.handlers.has(event)) {
      this.handlers.set(event, new Set());
    }
    this.handlers.get(event)!.add(handler);
  }

  async emit(event: string, data: any) {
    const handlers = this.handlers.get(event);
    if (!handlers) return;
    
    // Run all handlers concurrently, but wait for them
    await Promise.all(
      Array.from(handlers).map(h => h(data).catch(err => 
        console.error(`Handler failed for ${event}:`, err)
      ))
    );
  }
}

// contacts.service.ts
async createContact(input: CreateContactInput) {
  const contact = await this.db.contacts.create(input);
  
  // Sync to Elasticsearch
  await this.eventBus.emit('contact.created', contact);
  
  // For non-critical async work, enqueue
  await this.jobQueue.add('contact.webhook', { contactId: contact.id });
  
  return contact;
}

The in-process bus handles synchronous dependencies. The job queue handles async, non-critical ones. Kafka stays home.

Teams without ops capacity

This is the one nobody talks about in blog posts. Kafka requires:

• A dedicated broker (or managed service like AWS MSK, Confluent Cloud)
• ZooKeeper or KRaft controller (operational complexity)
• Monitoring for consumer lag, broker health, partition rebalancing
• Debugging tools and domain knowledge
• Someone on-call for broker failures

If your team has one backend engineer, one DevOps person, and two frontend developers, Kafka is not your friend. I've seen this at small SaaS startups. They were spending 20% of their ops time on Kafka health checks and lag monitoring.

A managed database (PostgreSQL, MySQL, even DynamoDB) with a simple polling consumer is radically simpler:

• One fewer operational dependency
• Easier to debug—everything is in your database
• Easier to monitor—just a table with a status column
• Easier to scale—usually just database scaling you already know

At my current role, we use AWS RDS for everything. Our job queue is a PostgreSQL table. Our dead-letter queue is another table. Operational overhead: near zero.

Simple request-reply interactions

If your system is mostly request-reply—client sends a request, server responds immediately—Kafka adds friction without benefit.

I've seen this in a ride-hailing prototype. Early version tried to emit every driver location change to Kafka, have passengers consume it. In practice:

• Passengers needed real-time updates, not batched events
• Kafka's throughput overhead was visible in latency
• WebSockets (or polling) was the actual requirement
• Kafka felt like extra baggage

If you need:

• Sub-100ms responses
• Request-reply patterns (RPC style)
• Real-time bidirectional communication

Then reach for WebSockets, gRPC, or HTTP. Kafka is for async, decoupled, high-throughput workloads. If that's not you, it's overhead.

Common mistakes

Mistake 1: "We'll need Kafka eventually." Don't build for scale you don't have. Start with what you need now. Refactoring from a job queue to Kafka later is trivial; staying on Kafka when you don't need it is painful.

Mistake 2: "We need Kafka for reliability." A job queue in a database is reliable. It's durable (it's in the database), retryable, and queryable. Kafka adds complexity, not reliability.

Mistake 3: "All our async work should use Kafka." No. Some async work is decoupled (publish-subscribe, multiple consumers, replay) and needs Kafka. Most async work is just "do this later" and a job queue suffices.

Mistake 4: "Kafka is simpler than managing a queue." It's not. Kafka is more powerful, but power costs operational budget.

When Kafka actually wins

To be clear: Kafka is the right choice when you have:

• High throughput (100k+ events/second)
• Multiple independent consumers with different interests
• Need for event replay or long-term retention
• Dedicated ops team to run it

At the e-commerce platform I mentioned, we do use Kafka for product catalog changes, inventory events, and order events—because 50+ services consume those events, they need replayed during deployments, and we have a platform team to run it.

But our email job queue? Still PostgreSQL. Our API responses? Still HTTP. Our driver location updates? Still WebSockets.

Key takeaways

• Kafka is powerful but not universal—most systems don't need it
• A job queue backed by your primary database handles low-volume async work cheaper and simpler than Kafka
• Internal, single-service events belong in an in-process event bus, not Kafka
• If your team lacks ops capacity, the operational cost of Kafka will hurt more than its features will help
• Request-reply patterns need WebSockets or HTTP, not Kafka
• Default to simple (database + worker). Move to Kafka only when simplicity becomes a bottleneck