Counting documents efficiently: best practices explained

Counting documents efficiently: best practices explained. Counts can be expensive.

Introduction

You're building a user dashboard that displays the total number of active listings, or you need to report analytics on how many orders have been processed this month. The instinct is to reach for a count operation—but counting documents efficiently in MongoDB is not always straightforward, and a naive approach can quickly become a performance bottleneck in production. Counting documents efficiently requires understanding when different count strategies are appropriate, how indexing affects performance, and when to trade real-time accuracy for speed.

This article covers the practical techniques we use across our production systems to keep count operations fast, from leveraging indexes to denormalizing counters where it matters most.

countDocuments with indexed filters

countDocuments() is the standard method for counting documents that match a filter. Unlike some legacy approaches, it respects collection filtering and is accurate—but performance depends entirely on whether your filter can use an index.

When you run countDocuments({ status: 'active' }), MongoDB must scan every document unless there's an index on the status field. If the index exists, MongoDB can count matching documents much faster by walking the index tree rather than examining the raw data.

Real-world scenario: An e-commerce platform tracks active seller listings. Sellers frequently check how many of their items are currently live.

// Without index: full collection scan
const activeCount = await db.collection('listings').countDocuments({
  seller_id: ObjectId('...'),
  status: 'active'
});

// With index on (seller_id, status), this becomes fast
db.collection('listings').createIndex({ seller_id: 1, status: 1 });

The compound index on seller_id and status means MongoDB can jump directly to documents matching both conditions. For a seller with 2 million listings but only 500 active, the index makes the difference between scanning 2 million documents and traversing a small index subset.

Always explain your count queries before production deployment:

const explanation = await db.collection('listings')
  .countDocuments({ seller_id: ObjectId('...'), status: 'active' })
  .explain('executionStats');

console.log('Examined documents:', explanation.executionStats.totalDocsExamined);
console.log('Returned count:', explanation.executionStats.nReturned);

// Ideally: totalDocsExamined ≈ nReturned (meaning index was used efficiently)

If totalDocsExamined is much higher than the returned count, you need a better index. If there's no index at all, MongoDB performs a COLLSCAN and examines every document—acceptable for small collections, dangerous at scale.

estimatedDocumentCount for whole collections

When you need the total count of an entire collection without any filter, estimatedDocumentCount() is faster than countDocuments({}) because it returns an estimate based on collection metadata rather than examining documents.

The tradeoff: estimatedDocumentCount() is approximate. It uses the collection's metadata to estimate the count, which means:

• It's nearly instant, regardless of collection size.
• The estimate can lag slightly behind the actual count, especially if many deletes have occurred.
• It returns zero for empty or non-existent collections.

Real-world scenario: A ride-hailing platform displays total rides taken on its homepage dashboard. Exact accuracy isn't critical—being off by 10 rides out of 50 million is acceptable.

// Fast but approximate
const estimatedRides = await db.collection('rides').estimatedDocumentCount();
console.log(`~${estimatedRides} rides have been completed`);

// Exact but slow on huge collections
const exactRides = await db.collection('rides').countDocuments({});
console.log(`${exactRides} rides have been completed`);

In a dashboard context, use estimatedDocumentCount() and refresh it periodically (e.g., every 5 minutes) rather than on every page load. If accuracy is critical—for instance, when calculating payouts to drivers—use countDocuments({}) with awareness that it may take longer.

Store counters for hot numbers

For metrics that are queried frequently and updated often, storing a denormalized counter in a separate document beats recounting every time. This pattern is especially valuable when you need real-time or near-real-time counts of high-velocity events.

Real-world scenario: A social platform tracks likes on posts. A popular post might accumulate thousands of likes in minutes. Recounting on every like would be expensive; instead, we maintain a running counter.

// Separate counters collection
const countersCollection = db.collection('counters');

// Initialize a counter
await countersCollection.updateOne(
  { _id: 'post_likes_12345' },
  { $set: { count: 0 } },
  { upsert: true }
);

// Increment on like
await countersCollection.updateOne(
  { _id: 'post_likes_12345' },
  { $inc: { count: 1 } }
);

// Retrieve count
const likeCounter = await countersCollection.findOne({ _id: 'post_likes_12345' });
console.log('Likes:', likeCounter.count);

This approach gives you:

• O(1) reads: Fetch the counter document, get the count instantly.
• O(1) writes: Atomic increment without recalculation.
• Eventual consistency: The count is always up-to-date within the same operation.

For even higher throughput, use a sharded counter pattern where you distribute increments across multiple counter documents and sum them when you need the total:

// Sharded counter: spread writes across N documents
const NUM_SHARDS = 10;

async function incrementShardedCounter(postId) {
  const shard = Math.floor(Math.random() * NUM_SHARDS);
  await countersCollection.updateOne(
    { _id: `post_likes_${postId}_shard_${shard}` },
    { $inc: { count: 1 } },
    { upsert: true }
  );
}

async function getShardedCount(postId) {
  const shards = await countersCollection
    .find({ _id: new RegExp(`^post_likes_${postId}_shard_`) })
    .toArray();
  
  return shards.reduce((sum, shard) => sum + shard.count, 0);
}

Sharding the counter reduces write contention and allows the platform to scale to millions of increments per second. The read cost is slightly higher (summing across shards), but for social platforms where reads are far less frequent than likes, this is a worthwhile tradeoff.

Avoid exact counts on huge result sets

When you have a large result set and you're tempted to count how many documents match a complex filter—especially across millions of records—consider whether you truly need the exact number.

Real-world scenario: A CRM needs to display how many customers have status='pending' and last_contact > 30 days ago. There are 50 million customers. A countDocuments() with a complex filter might take 10+ seconds.

// Expensive: scans 50 million docs or walks a massive index
const count = await db.collection('customers').countDocuments({
  status: 'pending',
  last_contact: { $lt: thirtyDaysAgo }
});

// Better: use aggregation with facet to sample or estimate
const pipeline = [
  {
    $match: {
      status: 'pending',
      last_contact: { $lt: thirtyDaysAgo }
    }
  },
  {
    $facet: {
      total: [{ $count: 'count' }],
      sample: [{ $sample: { size: 1000 } }]
    }
  }
];

const result = await db.collection('customers').aggregate(pipeline).toArray();
const totalEstimate = result[0].total[0]?.count || 0;

Or use pagination and only count the current page:

// Retrieve data with cursor, avoid global count
const query = {
  status: 'pending',
  last_contact: { $lt: thirtyDaysAgo }
};

const pageSize = 50;
const page = 1;

const docs = await db.collection('customers')
  .find(query)
  .skip((page - 1) * pageSize)
  .limit(pageSize + 1) // Fetch one extra to know if there's a next page
  .toArray();

const hasNextPage = docs.length > pageSize;
const results = docs.slice(0, pageSize);

console.log('Current page items:', results.length);
console.log('Has more:', hasNextPage);

By fetching one extra document, you can signal "there are more results" without knowing the total. This is often good enough for UI pagination and avoids the expensive count entirely.

Common mistakes

Using countDocuments({}) on billion-record collections: Developers often assume countDocuments is always fast. It's not—without an index, it's a full scan. Use estimatedDocumentCount() or cache the result instead.

Counting the same filter multiple times in a request: If your endpoint counts pending orders and then counts shipped orders, you're scanning the collection twice. Batch these into a single aggregation with $facet to scan once and get multiple counts.

Forgetting to index count filters: countDocuments({ user_id: X, status: 'active' }) without an index on those fields runs slow. Always verify your count queries use indexes via explain().

Denormalizing counters but not updating them: A counter document is only useful if it's kept in sync. If you store a likes count but fail to increment it atomically with each like, the counter becomes stale and misleading.

Counting in a loop: Calling countDocuments() inside a loop over many documents is inefficient. Use aggregation pipelines with $group to count groups in a single pass instead.

Key takeaways

• countDocuments() with indexed filters is fast and accurate—always verify your filter uses an index via explain().
• estimatedDocumentCount() is nearly instant for whole-collection counts and acceptable when slight inaccuracy is tolerable.
• Denormalized counters in separate documents provide O(1) reads and writes for frequently accessed metrics; use sharded counters for extreme write throughput.
• Avoid exact counts on huge result sets; instead use pagination signals, sampling, or cached estimates.
• Batch multiple counts into a single aggregation with $facet to minimize collection scans.
• Monitor count operation performance in production; slow counts are often a sign of missing indexes or data growth outpacing schema design.