WiredTiger storage engine basics: lessons from production

Most developers think of MongoDB as a black box: you insert documents, query them, and the database handles the rest. But understanding how MongoDB's WiredTiger storage engine actually stores and retrieves your data is the difference between a system that barely handles production load and one that scales reliably. We'll walk through the core mechanics that power millions of production queries every day, and show you how to tune your deployments based on what's actually happening under the hood.

Document-level concurrency: the foundation of performance

Before WiredTiger, MongoDB used a global lock for all writes. This meant that even if you had a cluster with hundreds of concurrent connections, only one could actually modify data at any time. WiredTiger changed this completely by implementing document-level concurrency control using optimistic concurrency control (OCC) with snapshots.

What this means in practice: when two connections write to different documents in the same collection simultaneously, they don't block each other. The storage engine maintains multiple versions of documents, allowing readers to see consistent snapshots without waiting for writers to finish. Writers only contend when modifying the exact same document.

Let's look at a real scenario. Suppose we have an e-commerce order system:

// Connection A updates order 12345
db.orders.updateOne(
  { _id: 12345 },
  { $set: { status: "shipped", shippedAt: new Date() } }
);

// Connection B updates order 12346 (different document)
db.orders.updateOne(
  { _id: 12346 },
  { $set: { status: "processing", updatedAt: new Date() } }
);

Both operations proceed in parallel because they touch different documents. The WiredTiger engine maintains separate version chains for each document. If Connection A had tried to update order 12345 again while Connection B was still modifying it, then contention would occur—but for different documents, you get genuine parallelism.

The downside: this doesn't eliminate all lock contention. Page-level locks still apply when the storage engine manages internal data structures. In high-concurrency workloads, you'll occasionally see write conflicts that require application-level retry logic. Transactions (introduced in MongoDB 4.0) added ACID guarantees across multiple documents, but they also introduced stronger locking semantics, so they're best reserved for operations that truly need atomic consistency across documents.

Compression for data and indexes: trading CPU for storage

Storage is expensive. Network bandwidth is expensive. But CPU is relatively cheap. WiredTiger's compression algorithms exploit this trade-off ruthlessly, compressing both document data and index entries on disk, then decompressing them in memory during access.

MongoDB uses Snappy compression by default for data and Prefix compression for indexes. Snappy trades slightly lower compression ratios for very fast decompression, which matters because every query that scans data must decompress it. Prefix compression on indexes works by storing only the differing suffix of keys within a page, dramatically reducing index size when keys have common prefixes (like timestamps or prefixed IDs).

The storage implications are substantial. A production system we tuned had 2.5TB of uncompressed data. With default Snappy compression, it dropped to 600GB on disk—a 76% reduction. That's not just saving space; it's reducing I/O bandwidth requirements, meaning your hardware can sustain higher query throughput.

Here's the trade-off in numbers. Let's say you have a collection of user documents averaging 2KB each, storing user activity logs:

In most production scenarios, the reduction in I/O dominates. Less I/O means lower p99 latencies, especially when your working set exceeds available RAM and you're hitting disk. The CPU overhead is minimal because modern CPUs decompress Snappy faster than the disk can deliver the compressed data.

You can change compression algorithms or disable it entirely, though we rarely recommend the latter:

// Configure compression when creating a collection
db.createCollection("largeEvents", {
  storageEngine: {
    wiredTiger: {
      configString: "block_compressor=snappy"
    }
  }
});

// Or for zstd (better ratio, slower decompression)
// configString: "block_compressor=zstd"

// Or disable compression (not recommended)
// configString: "block_compressor=none"

For most workloads, Snappy is the sweet spot. If your working set fits in RAM anyway, compression overhead barely matters. If you're I/O bound, compression usually helps significantly.

Internal cache sized at about half of RAM: the critical tuning parameter

WiredTiger maintains an internal cache separate from the OS page cache. This cache stores both data pages and index pages in uncompressed form. The size of this cache is one of the most important tuning decisions you'll make.

By default, MongoDB sizes the WiredTiger cache at:

• 50% of (total RAM - 1GB) on systems with more than 10GB of RAM
• max(256MB, (total RAM - 1GB) * 0.5) on smaller systems

This leaves room for the OS page cache, which caches the compressed data on disk. The two-tier approach is intentional: WiredTiger's cache holds hot, uncompressed data; the OS page cache (the other ~50% of RAM) caches compressed data from disk.

Let's work through a concrete example. You have a server with 64GB of RAM:

// Default calculation
// Available for caching: 64GB - 1GB = 63GB
// WiredTiger cache: 63GB * 0.5 = 31.5GB
// OS page cache: ~31.5GB

// In mongod logs, you'll see:
// "wiredTigerCacheSize": 31457280000

This is a reasonable default, but "reasonable" isn't "optimal." If your working set is 20GB, setting the cache to 31.5GB wastes memory—you could reduce it and use that extra RAM for application processes or other services. Conversely, if your working set is 50GB, a 31.5GB cache will cause frequent evictions, and you'd benefit from increasing it (if the machine has more RAM).

You can configure the cache explicitly in your MongoDB configuration:

// mongod.conf (YAML)
storage:
  wiredTiger:
    engineConfig:
      cacheSizeGB: 40  // Explicit 40GB cache

Monitor your actual cache performance using serverStatus:

db.serverStatus().wiredTiger.cache;

// Returns something like:
// {
//   "bytes currently in the cache": 28934847200,
//   "bytes dirty in the cache": 1234567800,
//   "eviction worker thread pages evicted": 5032,
//   "pages requested from cache that had to be read from disk": 1205,
//   "cache overflow pages read into cache": 342,
// }

If you see high "pages evicted" but low "pages read from disk," your cache is thrashing but the OS page cache is saving you. If "pages read from disk" is high relative to total queries, your cache is too small.

Checkpoints every 60 seconds: durability meets performance

WiredTiger doesn't write every change to disk immediately. Instead, it batches writes and creates checkpoints—atomic snapshots of the entire database—every 60 seconds by default. This dramatically improves write throughput because dirty pages stay in memory longer, allowing multiple writes to the same page to batch together before a single flush to disk.

Here's how it works: when you write a document, WiredTiger marks the relevant pages as dirty and updates an in-memory journal log. The journal is flushed to disk (relatively fast because it's sequential writes). Then, every 60 seconds, WiredTiger performs a checkpoint: it creates a consistent snapshot of all data on disk and writes a checkpoint record to the journal that says "everything as of this point is durable."

If the process crashes between checkpoints, MongoDB replays the journal on restart to recover any writes since the last checkpoint. This is why journal writes are durable but the full checkpoint is delayed.

The timeline looks like this:

• T=0s: Write document A (marked dirty, journaled)
• T=0.5s: Write document B to same page (batched, journaled)
• T=15s: Write document C to same page (batched, journaled)
• T=60s: Checkpoint triggered (all dirty pages flushed to disk, single atomic checkpoint record)
• T=61s: Next 60-second cycle begins

If a crash happens at T=45s, all three writes are recoverable from the journal. If it happens at T=61s, all three are already in the checkpoint and no recovery is needed.

You can adjust the checkpoint interval (though we rarely do):

// mongod.conf
storage:
  wiredTiger:
    engineConfig:
      checkpointSizeMB: 1000  // Checkpoint when 1GB of journal is used
      journalCompressor: snappy

More frequent checkpoints mean more durable data but higher I/O load. In high-write scenarios, checkpoint I/O can temporarily spike and cause latency blips. We've seen production incidents where a checkpoint coincided with peak traffic, causing p99 latencies to jump 5x for a few seconds. Monitoring checkpoint frequency is important:

db.serverStatus().wiredTiger.transaction;

// Watch "transaction checkpoints" to see how often they occur
// If checkpoints are rare but large, you may see periodic latency spikes

Common mistakes

• Setting cache size too large: Leaving insufficient memory for the OS page cache. If the OS starts swapping, you'll lose the performance benefits of the two-tier caching strategy.
• Disabling compression to reduce CPU: Most of the time, the I/O savings outweigh the CPU cost. Measure before disabling.
• Ignoring eviction stats: If WiredTiger is evicting pages faster than reads are accessing them, your queries are slow. This is a signal to increase cache size or reduce working set size (archive old data).
• Assuming checkpoint I/O doesn't matter: On systems with mixed read/write workloads, checkpoint spikes can affect p99 latencies. Monitor and plan for them.
• Running with journaling disabled: Journaling overhead is minimal relative to the durability guarantees it provides. Disabling it for performance is almost always the wrong trade-off.

Key takeaways

• Document-level concurrency allows parallel writes to different documents, but contention still occurs when multiple writes target the same document or page.
• Compression (default Snappy) reduces disk I/O by 60-80% with minimal CPU overhead; the OS page cache holds compressed data while WiredTiger's internal cache holds uncompressed data.
• The default cache size (50% of RAM) is a reasonable starting point, but monitor eviction rates and adjust based on your working set size.
• Checkpoints every 60 seconds batch writes for performance while the journal ensures recovery durability; checkpoint I/O can cause latency spikes in high-write scenarios.
• Understanding these mechanics helps you interpret mongostat output, diagnose latency issues, and make informed sizing decisions for your infrastructure.