Why replication factor 3

If you've spent any time running Kafka in production, you've probably heard the rule: use a replication factor of 3. It's not arbitrary—it's a carefully balanced choice that protects your data, keeps your infrastructure flexible, and prevents the kind of 2 AM incidents that haunt engineers. In this post, I'll walk you through exactly why replication factor 3 has become the standard for production topics, and what breaks when you deviate from it.

The math behind broker failures

Let's start with the fundamentals. A replication factor determines how many copies of each partition your data lives on across your Kafka cluster. With a replication factor of 3, every message you write exists on three separate brokers simultaneously.

Here's where it gets practical: if you have a 5-broker cluster and one broker goes down—whether due to a hardware failure, network partition, or an aggressive deployment—your data remains safe and accessible on the other two replicas. This is the critical insight: replication factor 3 survives one broker failure without data loss.

Compare this to replication factor 2: if one broker fails, you still have your data on one other broker. That sounds okay until you realize a second broker failure (which can happen during recovery, maintenance, or just bad luck) now causes permanent data loss. Or consider replication factor 1: the moment that single broker is unavailable, your partition is gone.

I learned this the hard way in an e-commerce platform I built. We started with replication factor 2 to save disk space, thinking we were being clever. The day a broker crashed and the replacement broker had a bad disk, we discovered our "redundancy" was more of a luxury than a guarantee. Switching to 3 meant committing to real protection.

The formula is simple: with replication factor N, you can tolerate N-1 broker failures. So replication factor 3 = tolerate 1 failure. Replication factor 4 = tolerate 2 failures, but now you're using 4x the disk space for the same logical data. Replication factor 3 is the sweet spot where you get meaningful fault tolerance without excessive waste.

Rolling upgrades without stopping the world

Production systems need maintenance. Kafka brokers need security patches, version upgrades, kernel updates on the host machines. The ability to do this without taking the cluster offline is invaluable.

Here's the scenario: you have a 3-broker cluster running Kafka 3.5. You need to upgrade to 3.6 to get a critical bug fix. You can't afford downtime. What do you do?

With replication factor 3, you can take down one broker, upgrade it, bring it back, wait for it to catch up, then move to the next one. As long as at least two replicas remain available during this window, your topic continues accepting reads and writes. Producers might see higher latency, but they don't see failures.

This is called a rolling upgrade, and it's essential for high-availability systems. I've used this pattern on a trading platform where we couldn't afford even a few seconds of downtime during market hours. Replication factor 3 made the upgrade process almost boring—we just cycled through brokers one at a time.

If you only had replication factor 2, you'd be much more nervous. Taking down one broker leaves exactly one replica, and if anything goes wrong during your upgrade—network hiccup, the upgraded broker fails to start, a second broker becomes unhealthy—you've now lost your topic entirely.

Replication factor 3 gives you a safety margin. If something unexpected happens during the upgrade of broker A, broker B and C are still there to serve traffic while you troubleshoot.

The power of min.insync.replicas=2

Here's where replication factor 3 becomes truly elegant: it works seamlessly with the min.insync.replicas (or min.isr) setting. Let me explain what this does first.

min.insync.replicas is a producer-level setting that tells Kafka: "Don't acknowledge my write as successful until at least N replicas have actually written it to disk." This is your guarantee against silent data loss.

Let's say you have a payment system where losing a single transaction is unacceptable. You might configure a topic like this:

typescript

// Topic configuration
{
  "num_partitions": 12,
  "replication_factor": 3,
  "config": {
    "min.insync.replicas": "2"
  }
}

// Producer configuration
{
  "acks": "all", // or "acks": "-1"
  "compression.type": "snappy"
}

Here's the guarantee you get: when your producer sends a message with acks=all (which waits for min.insync.replicas to acknowledge), the message isn't considered written until it's safely on 2 replicas. If the leader crashes immediately after, the message still exists on at least one follower replica. When a new leader is elected, your message is there.

The trio of settings—replication factor 3, min.insync.replicas 2, and acks=all—creates an unbreakable contract: you can lose one broker and still have all your data. You have real durability guarantees without checking every single replica (which would tank your throughput).

I've seen teams try to achieve this with replication factor 2 and min.insync.replicas 2, thinking "2 replicas are enough." The problem: if one replica crashes, you now have only 1 replica left, but min.insync.replicas still wants 2. Your producers start getting write failures. Your monitoring lights up. You're now in a degraded state during an unplanned failure, not a planned maintenance window. With replication factor 3, you still have 2 replicas left after one failure, so min.insync.replicas 2 keeps working.

Why replication factor 1 is only for local development

Sometimes I encounter teams using replication factor 1 in production, usually under the excuse of cost savings or simplicity. This is a category error. Replication factor 1 should only exist in your laptop, Docker compose files, or ephemeral test environments.

Here's why: with replication factor 1, a single broker failure is a data loss event. There are no copies elsewhere. Your partition is simply gone until you recover from backups—if you even have backups.

In a ride-hailing platform I helped migrate to Kafka, we initially kept a low-priority topic (driver location updates) at replication factor 1 to save disk costs. One broker crashed on a Tuesday morning. Our location tracking went dark for 90 minutes. We didn't lose critical business data, but the service degradation cost us. We immediately changed it to replication factor 3.

The cost of replication factor 3 is straightforward: you need 3x the disk space. In 2024, disk is cheap. The cost of being paged at 3 AM because a partition disappeared is not cheap. The business impact of stale or missing data is not cheap. Once you factor in operational overhead, replication factor 1 is a false economy.

I also see teams use replication factor 1 "temporarily" during a tight project deadline. Don't do this. Temporary always becomes permanent. By the time you mean to upgrade it, the code is in production, your team has moved on, and nobody remembers the technical debt. Just start with 3 from day one on anything that matters.

Practical considerations and trade-offs

I want to be honest: replication factor 3 isn't free, and there are scenarios where you might legitimately choose otherwise.

Disk space: Tripling your storage requirement is real. For high-throughput topics with retention measured in weeks, this can mean substantial infrastructure costs. One strategy: use replication factor 3 for critical topics (payments, orders, core events) and replication factor 2 for lower-criticality topics (analytics events, logs). Still not ideal, but a pragmatic trade-off.

Network overhead: Replication traffic counts against your network capacity. With 3 replicas, you're essentially multiplying your inter-broker traffic. For a 100 MB/sec topic, the cluster replicates an additional 200 MB/sec of data. Better network hardware or partitioning by geo-location can help.

Broker hardware: Three replicas mean every broker holds every partition's data. You need every broker to be reasonably healthy. A slow broker can drag down your entire cluster, because the leader waits for the slowest in-sync replica to acknowledge writes.

These aren't reasons to avoid replication factor 3; they're reasons to provision your Kafka infrastructure properly from the start.

Common mistakes

Setting replication factor 3 but min.insync.replicas 1: This gives you redundancy but not durability. Producers think their writes are safe when they're only on 1 replica. Use min.insync.replicas 2 minimum.
Upgrading an existing topic's replication factor without planning: Changing replication factor 3 on a large topic causes immediate network spike as Kafka rebalances replicas. Do this during off-peak hours and monitor your cluster closely.
Assuming replication factor 3 handles geo-distributed failures: All three replicas might still be in the same data center. Use rack awareness (broker.rack configuration) to spread replicas across physical racks, or better, across availability zones.
Using replication factor 3 without monitoring ISR (in-sync replicas): If your ISR drops below min.insync.replicas, producers will fail silently or with confusing errors. Alert on ISR size dropping below replication factor.

Key takeaways

Replication factor 3 survives exactly one broker failure without any data loss—the minimum threshold for production resilience.
Replication factor 3 enables zero-downtime rolling upgrades and maintenance by leaving two operational replicas during any single-broker downtime.
Pair replication factor 3 with min.insync.replicas=2 and acks=all to guarantee durable writes: one broker can fail and your data persists.
Replication factor 1 is acceptable only in local development, Docker environments, or throwaway test clusters—never in production.
The disk and network overhead of replication factor 3 is negligible compared to the operational cost of data loss or unplanned downtime.
Apply replication factor 3 consistently from day one; retrofitting it onto existing production topics is painful and risky.

Why replication factor 3: a hands-on guide for engineers