Monitor Node.js with Prometheus: lessons from production

Monitor Node.js with Prometheus: lessons from production

When a Node.js service starts consuming memory unexpectedly at 2 AM, or HTTP endpoints begin timing out under load, you need visibility into what's actually happening. Instrumenting your Node application with Prometheus isn't optional in production—it's your early warning system. In this guide, we'll walk through the patterns we've refined across e-commerce, SaaS, and ride-hailing platforms to instrument a Node service effectively.

Why Prometheus matters for Node.js

Node.js is single-threaded, which means the event loop health is everything. A blocked event loop can cause cascading failures across your entire service. Prometheus gives you the metrics needed to catch these issues before they become incidents. Unlike sampling-based tools, Prometheus collects every observation, which is critical when debugging subtle performance regressions in production.

Introducing prom-client — The standard Node library

prom-client is the de facto standard for Prometheus instrumentation in Node.js. It handles the heavy lifting: metric registration, aggregation, exposition format compliance, and memory efficiency. Most teams we've worked with adopt it from day one because reinventing metrics collection is a distraction.

Start by installing the package:

npm install prom-client

The library provides four metric types that map directly to Prometheus: Counter (monotonic increase), Gauge (can go up or down), Histogram (measures distribution), and Summary (percentile calculations). Understanding which type to use prevents data quality issues later.

Here's the minimal setup to expose metrics:

import client from 'prom-client'
import express from 'express'

const app = express()

// Metrics endpoint
app.get('/metrics', async (_req, res) => {
  res.set('Content-Type', client.register.contentType)
  res.end(await client.register.metrics())
})

app.listen(3000)

That single endpoint becomes your Prometheus scrape target. Grafana will pull from it every 15 seconds (or whatever interval you configure) to build your dashboards.

Default metrics — Event loop lag, memory and GC

The moment you call client.collectDefaultMetrics(), prom-client starts recording essential runtime telemetry without you writing a single additional line of instrumentation code. This is powerful and often overlooked.

import client from 'prom-client'

// Collect default Node.js metrics
client.collectDefaultMetrics({ timeout: 5000 })

// Metrics are now available at /metrics endpoint
// Includes: nodejs_eventloop_lag_seconds, nodejs_heap_size_bytes, etc.

What gets collected? The event loop lag histogram tells you when your single thread is blocked. The heap size metrics (used, total, external) expose memory growth patterns. Garbage collection metrics reveal if the GC is working overtime collecting garbage instead of executing your code.

We've caught several production issues using only these defaults:

• A memory leak in a scheduled job that wasn't being garbage collected
• An N+1 database query causing event loop stalls during peak traffic
• A third-party dependency leaking Node buffers on every request

Set the timeout parameter to match your Prometheus scrape interval. A 5-second collection window on a 15-second scrape prevents skewing the data.

HTTP histograms — Measure request duration by route and status

Default metrics are a foundation, but you need HTTP-level observability. Histograms let you measure request duration and answer questions like: which endpoints are slow? How does latency vary by status code? Are p99 times degrading over time?

Create a histogram for HTTP requests:

import client from 'prom-client'
import express from 'express'

const httpRequestDuration = new client.Histogram({
  name: 'http_request_duration_seconds',
  help: 'Duration of HTTP requests in seconds',
  labelNames: ['method', 'route', 'status_code'],
  buckets: [0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5]
})

const app = express()

// Middleware to track request duration
app.use((req, res, next) => {
  const start = Date.now()
  
  res.on('finish', () => {
    const duration = (Date.now() - start) / 1000
    httpRequestDuration.observe({
      method: req.method,
      route: req.route ? req.route.path : 'unknown',
      status_code: res.statusCode
    }, duration)
  })
  
  next()
})

app.get('/users/:id', (req, res) => {
  res.json({ id: req.params.id })
})

app.listen(3000)

The histogram buckets are critical. They define the boundaries Prometheus uses to calculate percentiles. We use buckets from 5ms (detecting fast endpoint regressions) to 5 seconds (catching hanging requests). Adjust based on your service's SLOs—a background job processor has different expectations than an API serving web traffic.

The middleware runs on every request, so measure early and finish after the response is sent. Capturing status code lets you distinguish between fast failures (4xx) and slow failures (5xx), which have different debugging paths.

Use route templates — /users/:id, not real ids

This is where teams often stumble. If you label metrics with the actual request path, you'll create thousands of time series:

// ❌ DO NOT DO THIS
route: req.path  // Results in /users/1, /users/2, /users/3, ... cardinality explosion

Instead, use the route template (parameter placeholders intact):

// ✅ CORRECT
route: req.route ? req.route.path : 'unknown'  // /users/:id for all user requests

High cardinality is the enemy of a healthy monitoring stack. One service with unbounded route labels can starve Prometheus memory and cause Grafana dashboards to timeout. The rule is simple: route labels should be in the dozens, not the millions.

Express provides the route template via req.route.path. If you're using a router, ensure you mount it early so this property is available in middleware. For dynamic routes you define at runtime, store the template explicitly:

app.get('/products/:productId/reviews/:reviewId', (req, res) => {
  // Middleware still sees req.route.path = '/products/:productId/reviews/:reviewId'
  res.json({})
})

Common mistakes

Not collecting default metrics early enough. Call collectDefaultMetrics() before your server starts listening. If you wait until after the app is running, you'll miss the initial memory and GC data.

Using unbounded label values. Beyond route cardinality, avoid labeling with user IDs, request IDs, or hostnames that vary. Every unique label combination creates a new time series. Prometheus has hard limits (usually 10k series per target), and you'll hit them faster than you expect.

Measuring request duration incorrectly. Don't measure from middleware entry to middleware exit—measure until the response is actually sent. Use the finish event on the response object to capture true end-to-end latency, including time spent serializing JSON or waiting for buffers to flush.

Forgetting to handle errors. If your metrics endpoint throws, Prometheus will fail silently or report an error during scrape. Wrap the /metrics handler in a try-catch and always return valid Prometheus exposition format.

Ignoring GC pressure in production. Default GC metrics are collected, but you need to understand them. Full GC pauses on a single-threaded runtime are genuinely painful. If you see GC happening every 10 seconds, that's a signal to reduce garbage creation (fewer object allocations, connection pooling, etc.).

Key takeaways

• Install prom-client and call collectDefaultMetrics() on startup—event loop lag and heap metrics catch 80% of production issues
• Expose a /metrics endpoint with the correct content type so Prometheus can scrape your service
• Add HTTP histograms with buckets tuned to your SLOs; use route templates (not real paths) to avoid cardinality explosion
• Measure request duration from middleware entry to response finish event, capturing status code for context
• Route labels should always use templates like /users/:id, never the actual path values
• Monitor GC metrics—frequent collections indicate memory pressure that will eventually cause outages