JSON vs Avro vs Protobuf: a hands-on guide for engineers

When JSON just isn't enough

You've built a Kafka pipeline that streams events from your orders service to analytics, billing, and inventory systems. Early on, everything works fine with JSON. Then the orders team adds a new field, and suddenly your analytics pipeline silently starts dropping data. A week later, your billing team is double-charging customers because they're parsing fields that no longer exist. This is where JSON vs Avro vs Protobuf: a hands-on guide for engineers becomes essential reading. Choosing a serialization format is one of the most consequential decisions you'll make for an event-driven system — it touches storage costs, debugging speed, team coordination, and backward compatibility.

Comparison: FORMAT vs TRADE-OFF

Every serialization format makes different bets. Understanding those trade-offs lets you pick the right one for your team's constraints.

JSON trades efficiency for simplicity. Avro trades tooling overhead for guarantees. Protobuf trades ease of initial setup for long-term scalability and generated client libraries.

JSON is fine to start

If you're building your first Kafka topic and need to ship something by Friday, JSON is the right call. It requires zero setup — no code generation, no schema registry, no learning curve. Your entire team can read an event in any text editor or kafka-console-consumer. That matters when you're debugging at 2 AM.

Here's a typical order event in JSON:

{
  "order_id": "ord_12345",
  "customer_id": "cust_789",
  "total_cents": 2999,
  "status": "confirmed",
  "items": [
    {
      "sku": "ABC-001",
      "quantity": 2,
      "price_cents": 1499
    }
  ]
}

The problem emerges over time. Six months in, the orders team adds shipping_address, another team adds tax_cents, and a third adds currency. Your consumers don't validate against a contract. The billing system silently ignores currency and assumes USD. The warehouse system crashes because it doesn't know what to do with shipping_address. You've now got a data quality disaster that nobody caught until customers complained.

This happens because JSON has no enforced schema. Anyone can add fields. Anyone can drop fields. There's no "version" — just gradual drift.

Binary formats save money

Once your system is mature and you're pumping millions of events per day through Kafka, bytes matter. A typical JSON order event (with whitespace) is roughly 350 bytes. The same event in Avro is about 80 bytes. In Protobuf, it's often 60 bytes.

For a team shipping 100 million events per day:

• JSON: 35 GB/day × 30 days = 1.05 TB/month in Kafka storage
• Avro: 8 GB/day × 30 days = 240 GB/month
• Protobuf: 6 GB/day × 30 days = 180 GB/month

That's not just storage cost — it's network bandwidth when consumers pull events, it's replication lag in your cluster, it's recovery time if a broker fails. The savings compound.

Here's the same order in Avro schema definition:

{
  "type": "record",
  "name": "Order",
  "fields": [
    { "name": "order_id", "type": "string" },
    { "name": "customer_id", "type": "string" },
    { "name": "total_cents", "type": "long" },
    { "name": "status", "type": "string" },
    {
      "name": "items",
      "type": {
        "type": "array",
        "items": {
          "type": "record",
          "name": "Item",
          "fields": [
            { "name": "sku", "type": "string" },
            { "name": "quantity", "type": "int" },
            { "name": "price_cents", "type": "long" }
          ]
        }
      }
    }
  ]
}

The schema is registered once in a schema registry. Producers and consumers both reference it by ID. The wire format stores only the ID and the field values — no field names, no type information, no duplication.

The schema is the real win

The true power of Avro, Protobuf, or any schema-driven format isn't compression — it's the contract. A schema is a legal agreement between your orders team and your analytics team. It says: "Here are the fields we promise to send. Here are their types. Here's what happens when we change them."

Avro's schema evolution is explicit. You can mark fields as optional (they have a default value), which means new fields don't break old consumers. You can add new fields to the end of a record safely. Old consumers reading new data simply ignore the extra fields. New consumers reading old data get the default value for fields that didn't exist yet.

Protobuf uses field numbers instead of field names. Field 1 is order_id forever. If you add currency, it becomes field 5. Old code doesn't know about field 5 and ignores it. New code knows about field 5 and uses it. The wire format doesn't care about field names — it only cares about numbers and types.

syntax = "proto3";

message Order {
  string order_id = 1;
  string customer_id = 2;
  int64 total_cents = 3;
  string status = 4;
  repeated Item items = 5;
  string currency = 6;  // New field, safe to add
}

message Item {
  string sku = 1;
  int32 quantity = 2;
  int64 price_cents = 3;
}

This schema stability is why large organizations standardize on Protobuf or Avro. When the orders team adds a field, the billing and analytics teams don't scramble for a hotfix. The systems continue working. You can roll out the change at whatever pace makes sense.

Common mistakes

Mistake 1: Choosing Protobuf to start. Protobuf is powerful but has overhead. You need a build step to generate code. You need to manage .proto files across repos. Teams just learning event-driven architecture often don't need this yet. JSON or Avro are better starting points.

Mistake 2: Running a schema registry without governance. If you set up Confluent Schema Registry for Avro but let anyone register any schema, you've added complexity without benefits. Schemas still drift. Establish a process: schemas are reviewed, versioned, and documented before they're registered.

Mistake 3: Mixing formats in a single system. One team uses JSON, another uses Avro, a third uses Protobuf. Now your consumers need three deserialization libraries. Your monitoring can't easily check schema compatibility. Pick one format and stick with it for the entire event layer.

Mistake 4: Forgetting about backwards compatibility in your application code. A schema says a field is optional, but your consumer code crashes if it's missing. You've got a schema contract that your code doesn't actually honor. Test what happens when fields are missing or extra.

Key takeaways

• Start with JSON if you're learning or moving fast. It's zero-overhead and debuggable. Just don't stay there once the system scales.
• Move to Avro or Protobuf when you have multiple teams producing and consuming events. The schema becomes your contract and prevents silent failures.
• Binary formats (Avro, Protobuf) save storage, bandwidth, and latency. At millions of events per day, the savings are substantial and worth the tooling investment.
• Protobuf gives you the smallest payload and the best cross-language tooling. Avro is easier to learn and has better Kafka integration out of the box.
• The real win isn't compression — it's the schema as a communication tool between teams. Backwards and forwards compatibility prevents breaking changes.
• Establish governance from day one. Schema reviews, versioning, and documentation matter more than the format itself.