Apache Kafka with Spring Boot: best practices explained

Introduction

Event-driven architectures have become essential in modern distributed systems, and Apache Kafka stands as the industry standard for building reliable, scalable event streaming platforms. Apache Kafka with Spring Boot: best practices explained covers the patterns and techniques our team has refined across high-volume e-commerce and SaaS platforms, where we process millions of events daily without losing a single order or payment record.

In this guide, we'll walk through producing events with ordered keys, consuming them safely in consumer groups, handling failures gracefully, and ensuring idempotent processing—the four pillars that keep event-driven systems running smoothly in production.

Produce — KafkaTemplate sends records with a key for ordering

When you send an event to Kafka, the broker assigns it to a partition. If you care about order—and in billing, inventory, and order processing you always do—you must provide a key. All events with the same key go to the same partition, preserving order for that entity.

Spring Boot makes this simple with KafkaTemplate. Rather than thinking of it as a generic message sender, think of it as your contract to preserve event sequence for critical entities.

@Service
public class OrderEventProducer {
  private final KafkaTemplate<String, OrderPlaced> template;

  public OrderEventProducer(KafkaTemplate<String, OrderPlaced> template) {
    this.template = template;
  }

  public void publishOrderPlaced(Order order) {
    OrderPlaced event = new OrderPlaced(
      order.getId(),
      order.getCustomerId(),
      order.getTotal(),
      Instant.now()
    );
    
    // Key is the order ID – all events for this order go to same partition
    template.send("orders", order.getId().toString(), event);
  }
}

Notice we're using order.getId().toString() as the key. This ensures that if the same order is modified multiple times, all those events land in the same partition and are processed in order. A consumer reading that partition will see OrderPlaced, then OrderShipped, then OrderDelivered—never out of sequence.

The template is generic over key and value types. If you're sending different event types, consider a common base class or union type. In our trading platform, we use a discriminated union approach with a type field, allowing one topic to carry multiple event shapes while remaining type-safe in Java.

You can also configure the template to wait for acknowledgment before returning:

@Bean
public ProducerFactory<String, OrderPlaced> producerFactory() {
  Map<String, Object> props = new HashMap<>();
  props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
  props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
  props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, JsonSerializer.class);
  props.put(ProducerConfig.ACKS_CONFIG, "all"); // Wait for all replicas
  return new DefaultProducerFactory<>(props);
}

@Bean
public KafkaTemplate<String, OrderPlaced> kafkaTemplate(
    ProducerFactory<String, OrderPlaced> factory) {
  return new KafkaTemplate<>(factory);
}

Setting acks=all means the broker waits for all in-sync replicas to acknowledge before telling you the send succeeded. It's slower but safer for data you cannot afford to lose.

Consume — @KafkaListener with a consumer group

On the other end, consumers read from topics. Spring Boot's @KafkaListener annotation handles subscription, deserialization, and error routing elegantly. The critical concept here is the consumer group: multiple instances of the same service can subscribe to the same topic under the same group ID, and Kafka will distribute partitions among them automatically.

@Service
public class BillingService {
  private final PaymentGateway gateway;

  public BillingService(PaymentGateway gateway) {
    this.gateway = gateway;
  }

  @KafkaListener(topics = "orders", groupId = "billing")
  public void onOrderPlaced(OrderPlaced event) {
    // Every order event lands here exactly once per consumer group
    gateway.chargeCustomer(event.getCustomerId(), event.getTotal());
  }
}

The groupId = "billing" is the key. If you have three instances of the billing service running, all three join the "billing" consumer group. Kafka's group coordinator divides the partitions among them. If one crashes, Kafka rebalances and redistributes the partitions to the survivors.

You can listen to multiple topics:

@KafkaListener(topics = {"orders", "returns"}, groupId = "billing")
public void onEvent(String payload) {
  // Handle both order and return events
}

Or filter by headers:

@KafkaListener(
  topics = "orders",
  groupId = "billing",
  filter = "orderFilter"
)
public void onOrderPlaced(OrderPlaced event) {
  // Only receives messages that pass orderFilter bean
}

The listener is invoked on a background thread pool. Keep the method fast—offload heavy work to another service or thread. If the method throws an exception, the error handler kicks in (we'll cover that next).

Handle failures — Configure a DefaultErrorHandler with retries and a dead-letter topic

Networks fail, downstream services go down, and bugs happen. Kafka consumers must handle this gracefully. Spring Kafka provides DefaultErrorHandler, which gives you backoff, retry, and dead-letter queuing in one configuration.

@Configuration
public class KafkaErrorHandlingConfig {

  @Bean
  public DefaultErrorHandler errorHandler() {
    // Retry with exponential backoff: 1s, 2s, 4s, 8s, 16s
    var backoff = new ExponentialBackOff(1000, 2.0);
    backoff.setMaxElapsedTime(60_000); // Stop after 60 seconds
    
    var handler = new DefaultErrorHandler(
      new DeadLetterPublishingRecoverer(kafkaTemplate()),
      backoff
    );
    
    return handler;
  }

  @Bean
  public KafkaTemplate<String, Object> kafkaTemplate() {
    return new KafkaTemplate<>(producerFactory());
  }
}

Here's what happens: if onOrderPlaced throws an exception, the message is retried after 1 second. If it fails again, it waits 2 seconds before retrying. This continues with exponential backoff until 60 seconds have elapsed. At that point, if the message still hasn't been processed, it's sent to a dead-letter topic (typically orders.DLT) for manual investigation.

Dead-letter topics are crucial. They're your "catch-all" for messages that consistently fail. Monitor them. When an order event lands in the dead-letter topic, it means something is genuinely broken—bad data, a corrupt event, or a critical downstream service failure. You need alerting and a playbook to handle it.

@Service
public class DeadLetterHandler {
  private final Logger logger = LoggerFactory.getLogger(DeadLetterHandler.class);

  @KafkaListener(topics = "orders.DLT", groupId = "dead-letter-handler")
  public void handle(ConsumerRecord<String, String> record) {
    logger.error("Dead-letter message: key={}, offset={}, value={}",
      record.key(), record.offset(), record.value());
    // Alert ops, store for replay
  }
}

You can also customize which exceptions trigger a retry and which should immediately go to the dead letter:

@Bean
public DefaultErrorHandler errorHandler() {
  var backoff = new ExponentialBackOff(1000, 2.0);
  backoff.setMaxElapsedTime(60_000);
  
  var handler = new DefaultErrorHandler(
    new DeadLetterPublishingRecoverer(kafkaTemplate()),
    backoff
  );
  
  // Don't retry validation errors; go straight to dead letter
  handler.addNotRetryableExceptions(ValidationException.class);
  
  return handler;
}

Idempotent consumers — Store processed event ids to handle at-least-once delivery

Kafka's guarantee is at-least-once delivery. If a consumer crashes after processing an event but before committing its offset, that event will be redelivered. This is by design—it prevents data loss—but it means you might process the same event twice.

In billing, processing an order twice means charging the customer twice. Unacceptable. The solution is idempotence: track which events you've already processed and skip duplicates.

@Service
public class IdempotentBillingService {
  private final PaymentGateway gateway;
  private final ProcessedEventRepository repository;

  public IdempotentBillingService(
      PaymentGateway gateway,
      ProcessedEventRepository repository) {
    this.gateway = gateway;
    this.repository = repository;
  }

  @KafkaListener(topics = "orders", groupId = "billing")
  public void onOrderPlaced(OrderPlaced event) {
    // Check if we've already processed this event ID
    if (repository.existsById(event.getOrderId())) {
      return; // Already processed, skip
    }
    
    // Process it
    gateway.chargeCustomer(event.getCustomerId(), event.getTotal());
    
    // Record that we've processed it
    repository.save(new ProcessedEvent(event.getOrderId()));
  }
}

The ProcessedEventRepository is typically a simple JPA entity backed by PostgreSQL or Redis. For very high throughput, Redis with a TTL is ideal—you only need to remember processed events for a sliding window, say 24 hours.

@Entity
@Table(name = "processed_events")
public class ProcessedEvent {
  @Id
  private String eventId;
  private Instant processedAt;

  public ProcessedEvent(String eventId) {
    this.eventId = eventId;
    this.processedAt = Instant.now();
  }
}

public interface ProcessedEventRepository extends JpaRepository<ProcessedEvent, String> {
}

Alternatively, if you're using Redis:

@Service
public class IdempotentBillingService {
  private final PaymentGateway gateway;
  private final RedisTemplate<String, String> redis;

  @KafkaListener(topics = "orders", groupId = "billing")
  public void onOrderPlaced(OrderPlaced event) {
    String key = "processed:" + event.getOrderId();
    
    // SET NX with expiration: set only if not exists
    Boolean isNew = redis.opsForValue().setIfAbsent(key, "1", Duration.ofHours(24));
    
    if (!isNew) {
      return; // Already processed
    }
    
    gateway.chargeCustomer(event.getCustomerId(), event.getTotal());
  }
}

The Redis approach is faster and more scalable because you're not hitting a relational database on every event. The tradeoff is that Redis keys expire, so if you need permanent audit history, you'll want both.

Common mistakes

• Forgetting the key. Without a key, events for the same entity can be processed out of order. Always provide a meaningful key tied to the entity you want to preserve order for.
• Blocking operations in the listener. If @KafkaListener methods make synchronous HTTP calls or database queries without connection pooling, you'll starve the thread pool. Use async clients or thread pools for heavy work.
• No monitoring of dead-letter topics. Dead letters are invisible by default. Set up alerts. When messages start appearing there, someone needs to know immediately.
• Assuming at-most-once delivery. Kafka gives you at-least-once. If you treat it as at-most-once, you'll miss data. Always implement idempotence for critical operations.
• Consumer group ID collisions. If you accidentally use the same group ID for two different consumer services, they'll steal partitions from each other. Use descriptive, unique group IDs like "billing", "shipping", "analytics".

Key takeaways

• Always provide a key when sending events with KafkaTemplate to preserve order for critical entities.
• Use @KafkaListener with a unique consumer group ID so multiple instances can scale horizontally.
• Configure DefaultErrorHandler with exponential backoff and dead-letter publishing to handle failures without losing events.
• Implement idempotent consumption by tracking processed event IDs; Kafka's at-least-once guarantee means you'll see duplicates.
• Monitor dead-letter topics closely—they indicate systemic issues that need immediate attention.
• Keep listener methods fast; offload heavy work to background threads or async services.