Producer settings that matter: best practices explained

Producer settings that matter

I've seen production outages caused not by Kafka itself, but by producers misconfigured to trade durability for speed, or batching settings that pile latency onto systems already struggling under load. The configuration behind reliable, fast producers isn't magic—it's a deliberate set of tuning decisions that sit between your application code and the broker cluster, and getting them right means the difference between a producer that loses messages silently and one that actually guarantees delivery.

When I started working with Kafka at scale, I learned quickly that the default producer settings are generous with throughput but stingy with reliability. Over the past few years building trading platforms, e-commerce pipelines, and real-time CRM systems, I've tuned these settings in dozens of production scenarios, and the same handful of parameters keep coming up as critical. This post walks through the four settings that matter most: how to wait for replicas, how to handle retries and timeouts, how to batch efficiently, and how to compress without regret.

acks=all — Wait for in-sync replicas. Required for durability.

The acks setting controls how many replicas must acknowledge a write before the producer considers it successful. It's perhaps the single most important decision you'll make about durability.

There are three choices:

• acks=0: Fire and forget. The producer doesn't wait for any acknowledgment. You'll get throughput measured in millions of messages per second, but you'll also lose messages whenever a broker restarts or a network hiccup occurs.
• acks=1: Wait for the leader. The leader broker acknowledges the write, but doesn't wait for replicas. If the leader crashes before replicating to followers, you lose that message.
• acks=all (or acks=-1): Wait for in-sync replicas. The leader waits until all in-sync replicas (ISR) have written the message to disk. This is the only setting that guarantees no message loss, assuming your cluster has at least two replicas per topic.

In a real-world scenario: I worked on a trading platform where orders flow through Kafka before hitting our database. A junior developer set acks=1 to squeeze a bit more throughput. Six months in, a broker crashed right after acknowledging an order but before replicating it. The order vanished. We lost the transaction and the customer's trust, and the incident cost hours of debugging and backfill work.

The performance difference between acks=all and acks=1 is real but manageable. You'll see latency increase by 10–50ms, depending on your network and replica count. That trade is worth taking for any system handling critical data.

Here's how to set it in your producer configuration:

const producer = kafka.producer({
  idempotent: true,
  transactionTimeout: 60000,
  maxInFlightRequests: 5,
});

const producerConfig = {
  acks: -1, // acks=all
  retries: Number.MAX_SAFE_INTEGER,
  maxInFlightRequests: 5,
  compression: 1, // gzip; we'll cover compression later
};

await producer.send({
  topic: 'orders',
  messages: [
    {
      key: order.customerId,
      value: JSON.stringify(order),
    },
  ],
  acks: -1,
});

Pro tip: pair acks=all with min.insync.replicas=2 on your broker side. This ensures the ISR must contain at least 2 replicas, preventing a scenario where a single replica crashes and you lose durability.

Retries and timeouts — delivery.timeout.ms bounds how long the producer keeps trying.

When a producer can't deliver a message—network timeout, broker unavailable, rate limit hit—it retries. The question is: how many times, and for how long?

Two settings work together here:

• retries: Number of times to retry a failed send. Default is 2147483647 (essentially unlimited).
• delivery.timeout.ms: Total time, in milliseconds, the producer will spend trying to deliver a message before giving up. Default is 120000ms (2 minutes).

The delivery.timeout.ms is the hard boundary. It includes the time spent waiting for acknowledgments, retrying, and backing off between attempts. Once this timer expires, the producer fails the send and calls your error handler.

Real-world scenario: I built a ride-hailing system where ride requests flow through Kafka to assignment engines. The ops team configured delivery.timeout.ms=5000 to fail fast, thinking it would keep latency low. But during peak traffic, brokers sometimes took 8–10 seconds to fully process a batch (due to compaction and I/O). Requests timed out, drivers didn't get assignments, and the service degraded. We increased the timeout to 30 seconds and saw stability return immediately.

Choose delivery.timeout.ms based on your use case:

• Critical data (payments, orders, trades): 30–60 seconds. Retries matter more than speed.
• Real-time data (analytics, metrics, events): 10–20 seconds. A lost event is acceptable; blocking the sender isn't.
• WebSocket messages to clients: 5–10 seconds. Keep the user experience responsive.

Configuration:

const producer = kafka.producer({
  idempotent: true,
  transactionTimeout: 60000,
  retries: 3,
  timeout: 30000, // delivery.timeout.ms equivalent in KafkaJS
});

// Or with lower-level control:
const producerConfig = {
  'delivery.timeout.ms': 30000,
  'retries': 3,
  'retry.backoff.ms': 100, // wait 100ms between retries
};

Also set retry.backoff.ms to avoid hammering a struggling broker. Start at 100ms and increase if you see patterns of cluster-wide slowness.

Batching — linger.ms and batch.size trade a few milliseconds of latency for throughput.

By default, the producer sends messages as soon as possible. But Kafka's real power comes from batching: waiting a few milliseconds to accumulate multiple messages into a single request, which the broker processes as one operation instead of many.

Two settings control this:

• linger.ms: How long to wait, in milliseconds, before sending a batch even if it's not full. Default is 0 (send immediately).
• batch.size: Maximum bytes per batch. Default is 16KB. Once a batch reaches this size, it's sent immediately, regardless of linger.ms.

The trade-off is simple: more lingering equals higher throughput but slightly higher latency per message.

Here's a concrete example from an e-commerce system I worked on: user events (views, clicks, cart additions) were flowing through Kafka to an analytics engine. With default settings (linger.ms=0), each event generated a separate request. We were handling about 50,000 events per second, but the brokers were under heavy load from all those small requests.

I changed the settings to linger.ms=10 and batch.size=32KB. Suddenly, 10 events arrived in a single batch instead of 10 separate requests. Broker CPU dropped by 40%, and end-to-end latency to the analytics dashboard improved because the broker spent less time context-switching. The trade-off? Each individual event now had an additional 10ms of latency. Acceptable for analytics, entirely unacceptable for a trading platform.

Recommended settings:

• High-throughput, low-latency-sensitive workloads (logs, metrics, events): linger.ms=10, batch.size=32KB
• Lower-throughput, latency-critical workloads (orders, payments): linger.ms=1, batch.size=16KB
• WebSocket broadcasts: linger.ms=5, batch.size=8KB (keep messages moving)

const producer = kafka.producer({
  idempotent: true,
  transactionTimeout: 60000,
  maxInFlightRequests: 5,
  // KafkaJS doesn't expose batch.size directly; it's handled internally
  // But you can control throughput with compression and max in-flight requests
});

// In raw librdkafka or Confluent Java client:
const producerConfig = {
  'linger.ms': 10,
  'batch.size': 32768, // 32KB
  'acks': -1,
  'compression.type': 'lz4',
};

Monitor broker metrics (incoming bytes per second, requests per second) to tune batch.size. If your brokers are CPU-bound, increase batching. If they're network-bound, it won't help much.

Compression — lz4 or zstd shrinks batches and network usage.

Compression is often overlooked, but it's one of the easiest wins for reducing network traffic and improving throughput.

Kafka supports four compression types:

• none: No compression. Baseline.
• gzip: Excellent compression (often 80–90% reduction), but high CPU cost. Not recommended for real-time systems.
• snappy: Fast compression, moderate reduction (40–60%). A solid default.
• lz4: Very fast, decent reduction (30–50%). Best for high-throughput, latency-sensitive systems.
• zstd: Modern codec, excellent compression (60–80%) with low CPU overhead. Becoming the standard.

Real scenario: A travel booking system I worked on was sending ~200MB/hour of booking events through Kafka. With no compression, the cluster was sending about 2GB/day to the network. I switched to compression.type=lz4. Network traffic dropped to 400MB/day, and producer latency barely increased (only 2–3ms extra per message for compression/decompression). That's a 5x improvement in network efficiency for almost no latency cost.

For most systems, I recommend:

• Default choice: lz4 or zstd. Both are fast and effective.
• CPU-constrained producers (mobile, IoT, edge): snappy.
• Storage-heavy systems (long retention, archival): zstd or even gzip (brokers handle decompression).

const producer = kafka.producer({
  idempotent: true,
  transactionTimeout: 60000,
  compression: 2, // 0=none, 1=gzip, 2=snappy, 3=lz4, 4=zstd
});

// Or declaratively:
const producerConfig = {
  'compression.type': 'lz4',
  'acks': -1,
  'linger.ms': 10,
  'batch.size': 32768,
};

Compression is especially effective when batching is enabled. Compressing 10 messages together is much more efficient than compressing them individually.

Common mistakes

• Setting acks=0 or acks=1 in production for critical data. You'll lose messages eventually. Always use acks=all for anything that matters.
• Ignoring delivery.timeout.ms and letting it default to 120 seconds. This can hide failures and make debugging harder. Set it explicitly based on your SLAs.
• Over-batching to squeeze throughput without measuring latency impact. A 100ms linger might give you a 20% throughput boost but ruin your p99 latency. Measure first.
• Using gzip compression for real-time systems. The CPU cost often outweighs the network savings. Stick with lz4 or zstd.
• Setting retries=0. Even transient network blips will fail messages. Use at least retries=3 with appropriate backoff.
• Forgetting to set idempotence on the producer. Without idempotent=true, retries can cause duplicate messages. Enable it whenever possible.

Key takeaways

• Always use acks=all for critical data. The latency cost (10–50ms) is tiny compared to the cost of losing a message.
• Set delivery.timeout.ms explicitly based on your workload: 30–60 seconds for critical data, 5–10 seconds for real-time.
• Batching (10–50ms linger time) can improve throughput by 2–10x with minimal latency impact. Measure before tuning.
• Compression with lz4 or zstd reduces network usage by 50–80% while adding only 2–5ms of CPU time per message.
• Pair your producer settings with broker-side tuning: min.insync.replicas=2, retention policies, and partition count matched to your throughput.
• Monitor broker metrics (CPU, network, request rates) to validate your settings in production. Theory and reality often diverge.