Analytics on MongoDB: a practical MongoDB & NoSQL guide

The Challenge: Analytics at Scale

Your MongoDB cluster is humming along—orders flowing, user sessions updating, inventory changing in real time. Then someone asks for a report: "How many orders did we process yesterday, broken down by region and payment method?" Before you know it, your application queries slow to a crawl because aggregation pipelines are scanning millions of documents while production traffic battles for resources. Analytics on MongoDB: a practical MongoDB & NoSQL guide covers strategies to run meaningful reports without sacrificing application performance.

The good news is that you don't have to choose between fast applications and useful analytics. We'll walk through proven patterns: replica set secondary reads, dedicated analytics nodes, precomputed aggregates, and exporting to specialized data warehouses. These aren't theoretical—they're what teams at scale actually do.

Secondary Reads: The First Line of Defense

MongoDB replica sets give you an immediate win: offload read-heavy analytics queries to secondary nodes. Secondaries replicate data from the primary with minimal lag (typically under a second) and don't participate in writes, so they're ideal for ad-hoc reporting.

To route analytics queries to a secondary, specify the read preference in your connection string or driver:

const client = new MongoClient(
  'mongodb+srv://user:pass@cluster.mongodb.net/mydb',
  {
    readPreference: 'secondaryPreferred'
  }
);

const analyticsDb = client.db('analytics');
const orders = analyticsDb.collection('orders');

// This query runs on a secondary, leaving primary untouched
const dailyRevenue = await orders.aggregate([
  { $match: { createdAt: { $gte: yesterday, $lt: today } } },
  { $group: { _id: '$region', total: { $sum: '$amount' } } }
]).toArray();

Be aware: secondaries lag behind the primary. If you query a secondary immediately after a write, you might not see that data yet. For reports that can tolerate eventual consistency—which most analytics can—this is a perfect fit. Set readPreference: 'secondaryPreferred' to fall back to the primary if no secondaries are available.

One caveat: if your reporting queries are truly heavy (scanning gigabytes, running complex aggregations), even a secondary will feel the load. That's when you need a dedicated analytics replica set member.

Dedicated Analytics Nodes

Add a secondary node to your replica set explicitly for analytics workloads. Configure it with lower priority so it never becomes primary, and tag it clearly in your topology:

Query the analytics node by tag:

const analyticsClient = new MongoClient(
  'mongodb+srv://user:pass@cluster.mongodb.net/mydb',
  {
    readPreference: 'secondary',
    readPreferenceTags: [{ role: 'analytics' }]
  }
);

// All queries go to the dedicated analytics secondary
const monthlyReport = await analyticsClient
  .db('mydb')
  .collection('orders')
  .aggregate([
    { $match: { month: 3 } },
    { $group: { _id: '$status', count: { $sum: 1 } } }
  ]).toArray();

This node can have different specifications than your production secondaries—perhaps lower CPU, more disk for indexes. The key benefit is isolation: your analytics queries never contend with production traffic.

Precompute Daily Aggregates

Heavy reporting doesn't have to happen on demand. Precompute results and store them in a dedicated collection. Run aggregation jobs during off-peak hours (perhaps 2 AM) and let applications query fast, denormalized results.

Create a scheduled job that runs nightly:

async function computeDailyAggregates() {
  const client = new MongoClient(process.env.MONGO_URI);
  const db = client.db('mydb');
  
  const yesterday = new Date();
  yesterday.setDate(yesterday.getDate() - 1);
  yesterday.setHours(0, 0, 0, 0);
  
  const today = new Date(yesterday);
  today.setDate(today.getDate() + 1);
  
  // Aggregate orders for yesterday
  const dailyStats = await db.collection('orders').aggregate([
    {
      $match: {
        createdAt: { $gte: yesterday, $lt: today }
      }
    },
    {
      $group: {
        _id: { region: '$region', status: '$status' },
        count: { $sum: 1 },
        revenue: { $sum: '$amount' },
        avgOrderValue: { $avg: '$amount' }
      }
    },
    {
      $out: 'daily_analytics'
    }
  ]).toArray();
  
  console.log(`Computed analytics for ${yesterday.toISOString()}`);
  await client.close();
}

// Run this job via cron or a task scheduler
// 0 2 * * * node computeDailyAggregates.js

Store results in a daily_analytics collection. Now your dashboards and reporting queries hit precomputed, indexed data instead of scanning raw orders:

// Fast query—already aggregated
const report = await db.collection('daily_analytics')
  .find({ _id: { region: 'US' } })
  .toArray();

This approach scales beautifully. You pay the cost of aggregation once per day, and your application gets instant results for thousands of queries. Trade-off: your reports are one day behind real time. For most business intelligence use cases, that's acceptable.

Export to a Data Warehouse for Heavy Analysis

If you're running complex joins across multiple collections, time-series analysis, or machine learning workloads, MongoDB isn't the right tool. Export your data to a data warehouse—Snowflake, BigQuery, Redshift—and let it shine.

Set up a nightly or hourly export pipeline. Here's a simple approach using mongodump and S3:

async function exportToWarehouse() {
  const { exec } = require('child_process');
  const fs = require('fs');
  const AWS = require('aws-sdk');
  
  const timestamp = new Date().toISOString();
  const dumpDir = `/tmp/mongo-dump-${timestamp}`;
  
  // Dump collections to local filesystem
  exec(
    `mongodump --uri="${process.env.MONGO_URI}" --out=${dumpDir}`,
    async (error) => {
      if (error) {
        console.error('Dump failed:', error);
        return;
      }
      
      // Upload to S3
      const s3 = new AWS.S3();
      const files = fs.readdirSync(dumpDir);
      
      for (const file of files) {
        const params = {
          Bucket: 'analytics-exports',
          Key: `mongo/${timestamp}/${file}`,
          Body: fs.readFileSync(`${dumpDir}/${file}`)
        };
        await s3.upload(params).promise();
      }
      
      console.log(`Exported to S3: analytics-exports/mongo/${timestamp}`);
    }
  );
}

Your warehouse can then ingest this data via native S3 connectors. Redshift Spectrum, BigQuery federated queries, and Snowflake's S3 integration all support this pattern. Once data is in the warehouse, you get SQL, time-window functions, and the ability to join with data from your application database, CRM, or ad platform.

For production systems, invest in proper ETL tooling: Fivetran, Stitch, or Kafka-based pipelines ensure data is consistent, deduped, and arrives on schedule. The initial setup takes time, but it's worth every hour.

Atlas Data Federation for S3 Queries

MongoDB Atlas offers a powerful alternative: Data Federation lets you query data stored in S3 directly from MongoDB, without importing it. This is ideal when you've exported large historical datasets or have data that lives outside MongoDB but you want to query alongside it.

Set up a federated database instance and point it to an S3 bucket containing JSON or CSV data:

const client = new MongoClient(process.env.MONGO_URI);
const federatedDb = client.db('federated');

// Query S3 data as if it's a MongoDB collection
const historicalOrders = await federatedDb.collection('s3_orders').find({
  createdAt: { $gte: new Date('2024-01-01') },
  region: 'EMEA'
}).toArray();

// Join federated S3 data with live MongoDB data
const enriched = await federatedDb.collection('orders').aggregate([
  { $match: { year: 2024 } },
  {
    $lookup: {
      from: 's3_orders',
      localField: 'customerId',
      foreignField: 'customerId',
      as: 'historical'
    }
  }
]).toArray();

Data Federation is especially useful for:

• Archiving old records to S3 while keeping hot data in MongoDB
• Joining real-time MongoDB data with historical data in object storage
• Querying partner datasets or third-party data lakes

The trade-off is performance: queries across the network to S3 are slower than local MongoDB queries. Use Federation for analytical workloads and occasional joins, not for latency-sensitive application queries.

Common Mistakes

Running heavy aggregations on the primary: Even if your primary has spare CPU, analytics queries lock resources needed for writes. Always route reporting elsewhere.

Ignoring index cost: Precomputed analytics need indexes just like any collection. If you're computing daily stats on region and status, create a compound index: db.daily_analytics.createIndex({ '_id.region': 1, '_id.status': 1 }). Without indexes, your "fast" precomputed queries become slow.

Treating secondaries as infinite resources: Each secondary has the same disk and memory as the primary. If your primary can't handle a query load, adding secondaries helps but doesn't solve the fundamental problem. Eventually you need to shard, export, or optimize.

Exporting stale data: If your export pipeline runs once a day at 2 AM, your warehouse is 22 hours behind by noon. Define SLAs for reporting freshness and align your architecture to meet them. For near-real-time analytics, consider Kafka or change streams feeding into your warehouse.

Key Takeaways

• Secondary reads with readPreference: 'secondaryPreferred' are the simplest first step—free isolation for most reporting workloads.
• Dedicated analytics nodes (tagged secondaries with priority 0) provide hard isolation for heavy aggregation jobs.
• Precompute daily or hourly aggregates in a separate collection—trade real-time precision for application performance.
• Export to a data warehouse (Snowflake, BigQuery) when you need joins across data sources or time-series analysis that SQL excels at.
• Atlas Data Federation bridges MongoDB and S3, enabling queries across hot and cold data without migration.
• Always use secondary reads for analytics—your production primary will thank you.
• Index your precomputed analytics collections the same way you'd index any other collection.