OpenTelemetry in Node.js: NestJS lessons from production

OpenTelemetry in Node.js

Distributed tracing feels like a luxury until your microservice architecture hits production and you can't figure out where a 5-second latency spike comes from. OpenTelemetry in Node.js gives you automatic visibility into every HTTP request, database query, cache hit, and message flowing through your system—without rewriting your code. Over 5+ years building platforms that process thousands of concurrent operations, I've learned that observability isn't optional: it's the difference between debugging for hours and pinpointing an issue in minutes.

In this article, we'll walk through setting up OpenTelemetry to trace across HTTP, Prisma, Redis, and Kafka. You'll see how auto-instrumentation handles the heavy lifting, why initialization order matters more than you'd expect, and how to layer custom spans on top for business-critical operations.

Why OpenTelemetry Matters in Production

Early in my e-commerce platform days, a customer complained that checkout was taking 8 seconds. My team added logging everywhere, redeployed, and waited for the issue to reproduce. When it did, we found the culprit: a single Redis HGETALL call on a hash with 50,000 fields. Without distributed tracing, we would've guessed for days.

OpenTelemetry lets you see the full request journey: incoming HTTP request → authentication service → product service → Kafka event → inventory update → cache invalidation. Every hop becomes visible. Combined with a backend like Jaeger or Grafana Tempo, you get flame graphs that show exactly where time is spent.

Auto-Instrumentation — The Node SDK Instruments HTTP, Express, Prisma and More

The beauty of OpenTelemetry's Node SDK is that it patches popular libraries at runtime. You don't manually wrap every database call or HTTP request. Instead, you install the right instrumentation packages and let them hook into the modules you already use.

Start with the core packages:

npm install @opentelemetry/api \
  @opentelemetry/sdk-node \
  @opentelemetry/sdk-trace-node \
  @opentelemetry/auto-instrumentations-node \
  @opentelemetry/exporter-trace-otlp-http

The @opentelemetry/auto-instrumentations-node package is a meta-package that installs instrumentation for Express, http, https, fs, pg, redis, nestjs, prisma, kafka, and dozens more. When your application starts, these instrumentations patch the native modules, so every HTTP request, database query, and cache operation automatically creates spans.

You typically don't need to install individual instrumentations—the meta-package handles it. But if you want fine-grained control, you can install and configure specific ones like @opentelemetry/instrumentation-prisma or @opentelemetry/instrumentation-redis.

Start It First — Load Tracing Before Any Other Import

This is the gotcha that trips up most developers. OpenTelemetry must initialize before you require any application code, because it works by monkey-patching modules at require-time. If Express is already loaded, the HTTP instrumentation won't patch it.

Create a dedicated tracing.ts file:

// src/tracing.ts
import { NodeSDK } from '@opentelemetry/sdk-node';
import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';

const otelSDK = new NodeSDK({
  traceExporter: new OTLPTraceExporter({
    url: process.env.OTEL_EXPORTER_OTLP_ENDPOINT || 'http://localhost:4318/v1/traces',
  }),
  instrumentations: [getNodeAutoInstrumentations()],
});

otelSDK.start();

process.on('SIGTERM', () => {
  otelSDK.shutdown()
    .then(() => console.log('OpenTelemetry shutdown'))
    .catch(err => console.error('Error shutting down OTel', err))
    .finally(() => process.exit(0));
});

export default otelSDK;

Then, in your entry point before any other code:

// src/main.ts
import './tracing'; // MUST be first
import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';

async function bootstrap() {
  const app = await NestFactory.create(AppModule);
  await app.listen(3000);
}

bootstrap();

Or in a CommonJS setup, use the --require flag:

node --require ./dist/tracing.js dist/main.js

Once initialized, every HTTP request hitting your Express or NestJS server automatically generates a root span. Every Prisma query nested under that request becomes a child span. Redis calls, Kafka publishes, and database connections all appear as spans in your trace.

Custom Spans — Wrap Important Business Operations for Clearer Traces

Auto-instrumentation gives you the infrastructure. Custom spans give you the narrative. When a customer tries to apply a discount code, you want to see a dedicated span showing validation logic, database lookup, and recalculation—not just raw SQL queries.

Inject the tracer and create spans around business logic:

// src/orders/order.service.ts
import { Injectable } from '@nestjs/common';
import { trace, context } from '@opentelemetry/api';

const tracer = trace.getTracer('order-service', '1.0.0');

@Injectable()
export class OrderService {
  async applyDiscount(orderId: string, code: string) {
    return tracer.startActiveSpan('apply-discount', async (span) => {
      span.setAttribute('order.id', orderId);
      span.setAttribute('discount.code', code);

      try {
        // Validate code
        await tracer.startActiveSpan('validate-code', async (validateSpan) => {
          validateSpan.setAttribute('code', code);
          // Your validation logic here
        });

        // Lookup discount in database
        const discount = await tracer.startActiveSpan('fetch-discount', async (fetchSpan) => {
          fetchSpan.setAttribute('code', code);
          return this.prisma.discount.findUnique({ where: { code } });
        });

        if (!discount) {
          span.recordException(new Error('Code not found'));
          span.setStatus({ code: 1 }); // ERROR
          throw new Error('Invalid code');
        }

        // Recalculate total
        const newTotal = await tracer.startActiveSpan('recalculate-total', async (recalcSpan) => {
          recalcSpan.setAttribute('discount.amount', discount.amount);
          return this.calculateTotal(orderId, discount.amount);
        });

        span.setAttribute('order.total.new', newTotal);
        return newTotal;
      } catch (error) {
        span.recordException(error as Error);
        span.setStatus({ code: 1 });
        throw error;
      }
    });
  }
}

Every span gets attributes (key-value data), status codes, and exception tracking. When you view this trace in Jaeger or Tempo, you see exactly where time went and where errors occurred. The visual hierarchy shows dependencies: apply-discount contains validate-code, fetch-discount, and recalculate-total.

For async operations across multiple services, context propagation ensures child spans link back to the root. When an HTTP request triggers a Kafka message, and that message spawns a database write, all three operations share the same trace ID.

Export Anywhere — OTLP to Grafana Tempo, Jaeger, Honeycomb or Datadog

The OTLP (OpenTelemetry Protocol) exporter in our tracing setup sends traces as HTTP POST requests. Any backend that speaks OTLP can receive them. The beauty is that you write the instrumentation once and swap backends without code changes.

For local development, start Jaeger with Docker:

docker run -d \
  -e COLLECTOR_OTLP_ENABLED=true \
  -p 16686:16686 \
  -p 4318:4318 \
  jaegertracing/all-in-one:latest

Set the environment variable and your traces appear at http://localhost:16686:

OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318/v1/traces node src/main.js

For production, many teams use Grafana Tempo (open-source, runs in Kubernetes) or commercial solutions like Honeycomb and Datadog. Tempo's endpoint looks like:

OTEL_EXPORTER_OTLP_ENDPOINT=https://tempo.monitoring.svc.cluster.local:4318/v1/traces

Datadog's exporter is slightly different and requires an API key, but the pattern is identical. The point: instrument once, export anywhere.

Common Mistakes

• Forgetting to import tracing first. If you load tracing after Express or Prisma, the monkey-patches don't stick. Always check that your entry point imports tracing before any application code.
• Creating too many spans. A span per line of code creates noise. Span at the business operation level (authenticate user, process payment, send email). Let auto-instrumentation handle infrastructure.
• Not setting span attributes. Spans without attributes are hard to filter. Always add user ID, resource IDs, operation names, and error details so you can query traces later.
• Ignoring context propagation. If you fire off async tasks or Kafka messages, ensure you use OpenTelemetry context APIs so child operations link to the parent trace. Don't spawn tasks with bare async/await without wrapping them.
• Sampling too aggressively. If you sample only 1 in 10,000 traces, you'll miss rare errors. For production, sample high-volume endpoints but keep errors and slow requests 100%.

Key Takeaways

• OpenTelemetry auto-instrumentation patches HTTP, database, and cache libraries at runtime, giving you visibility with minimal code.
• Always load tracing before any other imports so monkey-patches apply to your dependencies.
• Layer custom spans over auto-instrumentation to trace business logic, not just infrastructure.
• Use OTLP to export traces to any backend: Jaeger, Tempo, Honeycomb, Datadog. Instrument once, swap backends freely.
• Observability compounds: small traces from every request build a picture of system behavior that manual logging cannot match.

After using distributed tracing on ride-hailing and trading platforms, I can't imagine building production systems without it. The first time you catch a cascading failure because a single Kafka consumer fell behind, or a Redis key explosion is slowing queries, you'll understand why.