Kafka security: a Kafka & WebSockets guide for developers

By day 42 of any streaming architecture, you've likely moved past hello-world Kafka examples and are running multiple services in production. That's when security stops being optional. Kafka security: a Kafka & WebSockets guide for developers isn't about checking a compliance box—it's about stopping your consumer groups from reading the wrong topics, preventing man-in-the-middle attacks on message streams, and being able to trace which service changed a broker configuration at 3am.

In this guide, we'll walk through four layers of Kafka security that matter most when you're connecting WebSocket clients to Kafka brokers: TLS encryption, SASL authentication, ACLs, and audit logging. We'll show you how they stack together and where developers most commonly go wrong.

TLS: Encrypt traffic between clients and brokers

TLS is your first line of defense. Without it, every message between your client (whether it's a Node.js service, a WebSocket gateway, or a consumer) and your Kafka broker travels in plain text. Anyone on the network—a developer in a shared office, a compromised pod in your Kubernetes cluster, or someone with packet capture tools—can read your data.

Setting up TLS in Kafka means:

• Generating or obtaining certificates for your brokers
• Configuring brokers to listen on a TLS port (usually 9093)
• Having clients trust the broker's certificate

Here's how to configure a Kafka broker for TLS:

// broker configuration (typically in server.properties or docker-compose.yml)
listeners=PLAINTEXT://localhost:9092,SSL://localhost:9093
advertised.listeners=PLAINTEXT://localhost:9092,SSL://localhost:9093
ssl.keystore.location=/etc/kafka/secrets/kafka.server.keystore.jks
ssl.keystore.password=broker-keystore-password
ssl.key.password=broker-key-password
ssl.truststore.location=/etc/kafka/secrets/kafka.server.truststore.jks
ssl.truststore.password=broker-truststore-password

On the client side (your NestJS backend connecting to Kafka):

import { Kafka } from 'kafkajs';
import fs from 'fs';

const kafka = new Kafka({
  clientId: 'websocket-gateway',
  brokers: ['kafka-broker-1:9093', 'kafka-broker-2:9093'],
  ssl: {
    rejectUnauthorized: true,
    ca: [fs.readFileSync('/etc/kafka/secrets/ca-cert', 'utf-8')],
    key: fs.readFileSync('/etc/kafka/secrets/client.key', 'utf-8'),
    cert: fs.readFileSync('/etc/kafka/secrets/client.cert', 'utf-8'),
  },
});

In a containerized environment (Docker or Kubernetes), mount your certificates as secrets and reference the paths. Don't hardcode certificates or embed them in images.

One critical point: TLS encrypts traffic but doesn't verify who is making the connection. A valid client certificate proves the connection is encrypted, but not authenticated. That's where SASL comes in.

SASL authentication: SCRAM, OAuth, or IAM

SASL (Simple Authentication and Security Layer) answers the question: "Who are you?" It runs on top of TLS and confirms the identity of the client before allowing it to produce or consume messages.

The most common choices for Kafka are:

• SCRAM-SHA-256/512: Kafka stores a salted hash of the client's password. Simple to set up, no external system needed. Good for internal services.
• OAuth2: Integrate with an OAuth provider (Auth0, Okta, cloud provider). Better for multi-team environments where you want centralized identity.
• IAM (AWS, Azure, GCP): Use your cloud provider's identity system. Reduces credential sprawl in Kubernetes or Lambda environments.

Here's SCRAM-SHA-256 setup on the broker side:

// In broker config
sasl.enabled.mechanisms=SCRAM-SHA-256
listeners=SASL_SSL://localhost:9093
advertised.listeners=SASL_SSL://localhost:9093
security.inter.broker.protocol=SASL_SSL

// Create a user via Kafka CLI (or automated in your deployment)
// bin/kafka-configs.sh --bootstrap-server localhost:9093 \
//   --alter \
//   --add-config 'SCRAM-SHA-256=[password=my-secure-password]' \
//   --entity-type users \
//   --entity-name websocket-gateway

And on your NestJS client:

const kafka = new Kafka({
  clientId: 'websocket-gateway',
  brokers: ['kafka-broker-1:9093'],
  ssl: {
    rejectUnauthorized: true,
    ca: [fs.readFileSync('/etc/kafka/secrets/ca-cert', 'utf-8')],
  },
  sasl: {
    mechanism: 'scram-sha-256',
    username: process.env.KAFKA_USERNAME,
    password: process.env.KAFKA_PASSWORD,
  },
});

For OAuth2, Kafka 3.2+ supports OAUTHBEARER. You'd provide a callback that fetches a JWT from your OAuth provider and refreshes it before expiration.

Whatever mechanism you choose, rotate credentials regularly. In a Kubernetes environment, update the secret and restart pods. For long-lived applications (like a WebSocket gateway), implement credential rotation without dropping active connections.

ACLs: Each service reads and writes only its topics

Even with TLS and SASL, any authenticated user can still produce and consume from any topic. ACLs (Access Control Lists) enforce least privilege: the payments service can only write to payments.events, the notifications service can only read from it.

A typical ACL rule says: "User X can perform operation Y on resource Z."

// Example ACLs for a real-world setup

// WebSocket gateway reads from user events (doesn't produce)
// bin/kafka-acls.sh --bootstrap-server localhost:9093 \
//   --add \
//   --allow-principal User:websocket-gateway \
//   --operation Read \
//   --topic user-events \
//   --group websocket-gateway-group

// Payments service writes to payments.events and reads from payment-commands
// bin/kafka-acls.sh --bootstrap-server localhost:9093 \
//   --add \
//   --allow-principal User:payments-service \
//   --operation Write \
//   --topic payments.events

// bin/kafka-acls.sh --bootstrap-server localhost:9093 \
//   --add \
//   --allow-principal User:payments-service \
//   --operation Read \
//   --topic payment-commands \
//   --group payments-service-group

// Describe cluster (often needed for health checks)
// bin/kafka-acls.sh --bootstrap-server localhost:9093 \
//   --add \
//   --allow-principal User:payments-service \
//   --operation DescribeCluster

Enable ACL enforcement on your brokers:

// broker config
authorizer.class.name=org.apache.kafka.security.authorizer.AclAuthorizer
super.users=User:admin

Start restrictive. A service needs Read on a topic and Read on its consumer group. Don't grant All permissions. Many teams make the mistake of setting up ACLs only after incidents, not proactively.

Audit: Track who created topics and changed configs

Encryption, authentication, and authorization prevent unauthorized access. Audit logs tell you what authorized users actually did. When a critical topic gets deleted at 2am, or a broker configuration changes unexpectedly, audit trails answer: who, when, and what.

Enable broker audit logs:

// broker config
log4j.appender.auditAppender=org.apache.log4j.RollingFileAppender
log4j.appender.auditAppender.File=logs/audit.log
log4j.appender.auditAppender.layout=org.apache.log4j.PatternLayout
log4j.appender.auditAppender.layout.ConversionPattern=%d{ISO8601} %p %m%n
log4j.logger.kafka.authorizer.logger=INFO, auditAppender

// Also capture broker config changes
broker.log.cleanup.policy=delete
// or use Kafka's event streaming (broker sends ConfigChange events)

Parse and centralize these logs. In production, pipe them to your observability stack (ELK, Datadog, CloudWatch):

// Example: tail audit logs and send to a logging service
import * as fs from 'fs';
import * as readline from 'readline';

const auditStream = fs.createReadStream('logs/audit.log');
const rl = readline.createInterface({
  input: auditStream,
  crlfDelay: Infinity,
});

rl.on('line', (line) => {
  const auditEvent = parseAuditLog(line);
  // Send to CloudWatch, Datadog, or your logging backend
  logger.info('kafka_audit', { event: auditEvent });
});

Key events to monitor: topic creation/deletion, ACL changes, broker configuration updates, and failed authentication attempts. Set up alerts when unusual patterns occur (e.g., someone deleting a production topic).

Common mistakes

• Using TLS without SASL: Encryption alone doesn't authenticate. Any client that can reach the broker can still connect and read all topics. Combine them.
• Overly permissive ACLs: Granting All permissions to multiple services defeats the purpose. Start with the minimum and expand only when necessary.
• Hardcoding credentials: Never embed SASL passwords in your code or Docker images. Use environment variables, Kubernetes secrets, or a secrets manager.
• Ignoring certificate expiration: Certificates have expiry dates. Set up monitoring and automate renewal before they expire, or your clients will suddenly fail to connect.
• No audit logging: If you can't trace who made a change, you can't investigate incidents or enforce accountability. Enable auditing from day one.
• Rotating credentials without coordination: If you update a SASL password but don't coordinate with the service that uses it, traffic stops. Plan credential rotation as a team.

Key takeaways

• TLS encrypts the channel but doesn't authenticate. Always pair it with SASL (SCRAM, OAuth, or IAM).
• ACLs enforce least privilege: each service should only read and write the topics it needs.
• Audit logs are your incident-response lifeline. Centralize them and alert on suspicious activity.
• Store credentials in secrets managers or environment variables, never in code or images.
• Test your security setup in staging before production. A misconfigured ACL that blocks legitimate traffic is as bad as no ACLs at all.
• Security is layered. TLS + SASL + ACLs + auditing together form a strong defense; any one layer alone is incomplete.