Monitoring third-party dependencies: a step-by-step guide

Monitoring third-party dependencies: a step-by-step guide

Your application depends on external services—payment processors, shipping APIs, analytics platforms, map services—but you often treat them as black boxes until they fail. By the time your customer support team hears about it, you've already lost transactions or service quality. This guide walks you through monitoring third-party dependencies: a step-by-step guide that puts visibility first, so you know when partners fail before your users do.

Why monitoring outbound calls matters

Third-party dependency failures cascade quickly. A slow payment gateway doesn't just delay one checkout; it degrades your entire user experience, increases timeout errors, and can trigger cascading failures in your own system. Without instrumentation, you're flying blind. You need to know latency, error rates, and availability of every external call your application makes.

We've all experienced the scenario: support escalates a ticket, you check logs, and only then discover that a partner API has been degrading for twenty minutes. By instrumenting outbound calls with Prometheus metrics, you shift from reactive troubleshooting to proactive alerting. You catch issues before they become incidents.

Measure outbound call latency and errors

Start by instrumenting every outbound HTTP call—whether it's a REST API, gRPC service, or webhook. The goal is to capture two critical signals: how long calls take and whether they fail.

Here's a practical approach using a middleware or interceptor pattern. In a Node.js/NestJS stack, we wrap HTTP client calls with Prometheus instrumentation:

import { Injectable } from '@nestjs/common';
import { HttpService } from '@nestjs/axios';
import { Counter, Histogram, register } from 'prom-client';

const httpRequestDuration = new Histogram({
  name: 'http_outbound_request_duration_seconds',
  help: 'Duration of outbound HTTP requests',
  buckets: [0.05, 0.1, 0.25, 0.5, 1, 2.5, 5],
  labelNames: ['dependency', 'method', 'status'],
});

const httpRequestErrors = new Counter({
  name: 'http_outbound_request_errors_total',
  help: 'Total outbound HTTP request errors',
  labelNames: ['dependency', 'method', 'error_type'],
});

@Injectable()
export class InstrumentedHttpService {
  constructor(private http: HttpService) {}

  async get<T>(
    url: string,
    dependency: string,
  ): Promise<T> {
    const startTime = Date.now();
    try {
      const response = await this.http.get<T>(url).toPromise();
      const duration = (Date.now() - startTime) / 1000;
      httpRequestDuration
        .labels(dependency, 'GET', response.status)
        .observe(duration);
      return response.data;
    } catch (error) {
      const duration = (Date.now() - startTime) / 1000;
      const errorType = error.code === 'ECONNREFUSED' 
        ? 'connection_refused' 
        : error.response?.status || 'unknown';
      httpRequestErrors
        .labels(dependency, 'GET', String(errorType))
        .inc();
      httpRequestDuration
        .labels(dependency, 'GET', 'error')
        .observe(duration);
      throw error;
    }
  }
}

In a .NET environment, HttpClientFactory with a delegating handler achieves the same result:

public class PrometheusHttpMessageHandler : DelegatingHandler
{
    private readonly IMetricsCollector _metrics;
    private readonly string _dependencyName;

    public PrometheusHttpMessageHandler(
        IMetricsCollector metrics, 
        string dependencyName)
    {
        _metrics = metrics;
        _dependencyName = dependencyName;
    }

    protected override async Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request, 
        CancellationToken cancellationToken)
    {
        var stopwatch = Stopwatch.StartNew();
        try
        {
            var response = await base.SendAsync(request, cancellationToken);
            stopwatch.Stop();
            _metrics.RecordOutboundRequestDuration(
                _dependencyName, 
                request.Method.ToString(), 
                (int)response.StatusCode, 
                stopwatch.Elapsed.TotalSeconds);
            return response;
        }
        catch (Exception ex)
        {
            stopwatch.Stop();
            _metrics.RecordOutboundRequestError(
                _dependencyName, 
                request.Method.ToString(), 
                ex.GetType().Name);
            throw;
        }
    }
}

The key insight: capture duration for both success and failure cases, and always label by dependency name. This separation lets you alert on specific partners, not lumping all external calls together.

Label by dependency name

Labels are your friend. Without them, you can't distinguish between a Stripe outage and a Twilio degradation. Every outbound call should carry a dependency label that identifies the partner service.

Establish a naming convention early. Good practice is to use a short, consistent identifier:

• stripe_payments for Stripe's charge API
• sendgrid_email for SendGrid's send endpoint
• mapbox_geocoding for Mapbox geocoding service
• aws_s3 for S3 object operations

Include additional labels that provide context: method (GET, POST), status (200, 429, 500), and error_type (timeout, connection_refused, rate_limit). This granularity lets you write precise alert rules later.

In your Prometheus scrape config, ensure you're collecting from your application's metrics endpoint:

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'api_server'
    static_configs:
      - targets: ['localhost:3000']

Then in Prometheus, query your metrics with labels. For example, to see average latency by dependency:

rate(http_outbound_request_duration_seconds_sum[5m]) 
/ 
rate(http_outbound_request_duration_seconds_count[5m])

Alert on sustained failures

Metrics alone don't prevent incidents. You need alerting rules that fire when dependencies degrade. The trick is distinguishing between transient hiccups and real problems. A single 500 error is noise; a pattern of errors over five minutes is actionable.

Define alert rules in Prometheus that measure both error rate and latency degradation. Here's a practical alerting strategy:

groups:
  - name: dependency_health
    interval: 30s
    rules:
      - alert: DependencyHighErrorRate
        expr: |
          (
            rate(http_outbound_request_errors_total[5m]) 
            / 
            rate(http_outbound_request_duration_seconds_count[5m])
          ) > 0.05
        for: 5m
        annotations:
          summary: "Dependency {{ $labels.dependency }} has high error rate"
          description: "Error rate is {{ $value | humanizePercentage }} over 5m"

      - alert: DependencyHighLatency
        expr: |
          histogram_quantile(0.95, 
            rate(http_outbound_request_duration_seconds_bucket[5m])
          ) > 2
        for: 5m
        annotations:
          summary: "Dependency {{ $labels.dependency }} p95 latency is {{ $value }}s"

      - alert: DependencyTimeout
        expr: |
          rate(http_outbound_request_errors_total{error_type=~"timeout|ETIMEDOUT"}[5m]) > 0.1
        for: 2m
        annotations:
          summary: "{{ $labels.dependency }} is timing out"

The for: 5m clause is critical—it prevents alert fatigue from transient blips. An error rate spike lasting 30 seconds won't trigger an alert; sustained degradation will.

Route these alerts to your incident channel (PagerDuty, Slack) so the team gets notified immediately. Include the dependency name and metric value in the alert; engineers need to know exactly which partner is failing and how badly.

Show dependency health on dashboards

Alerts wake you up at 3 AM, but dashboards let you understand the full picture during troubleshooting. Build a dashboard that shows dependency health at a glance.

Use Grafana to visualize latency, error rates, and availability by dependency. Here's a recommended layout:

1. Status heatmap: A table showing each dependency with its current status (green/yellow/red), based on error rate and latency thresholds.
2. Latency trend: A line graph with p50, p95, p99 latency over time for each dependency.
3. Error rate over time: A bar or line chart showing errors per dependency, segmented by error type.
4. Request volume: Total requests per dependency per minute to spot traffic shifts.

In Grafana, create a single-stat or gauge panel to show current dependency health:

SELECT
  dependency,
  (
    rate(http_outbound_request_errors_total[5m]) 
    / 
    rate(http_outbound_request_duration_seconds_count[5m])
  ) as error_rate,
  histogram_quantile(0.95, 
    rate(http_outbound_request_duration_seconds_bucket[5m])
  ) as p95_latency
FROM
  prometheus
WHERE
  error_rate > 0.01 OR p95_latency > 1

Add a table panel listing all dependencies sorted by error rate or latency. Use color-coding: green for healthy (<1% errors, <500ms p95), yellow for degraded (1-5% errors, 500ms-2s p95), red for critical (>5% errors, >2s p95).

This dashboard becomes your command center during an incident. You can instantly see which dependency caused a cascade, when it started failing, and how long it's been degraded.

Common mistakes

Not instrumenting all outbound calls: Teams often instrument the obvious dependencies (payment processors) but miss internal API calls or third-party logging services. If a service makes outbound calls, instrument them all.

Using vague dependency names: Labeling all calls to AWS as a single dependency masks which service (S3, SQS, RDS) is failing. Use specific, granular names that map 1:1 to the external service and operation.

Ignoring timeouts as errors: Timeouts are errors, not a separate category. They degrade user experience as much as a 500 response. Always include timeout/connection errors in your error rate calculations.

Setting alert thresholds without baseline data: Don't guess at alert thresholds. Let your system run for a week, observe normal behavior, then set thresholds based on what's actually abnormal for each dependency.

Forgetting to measure call volume: A dependency with a sudden traffic spike might show higher latency even if the partner service is healthy. Track request volume alongside latency so you can correlate patterns.

Key takeaways

• Instrument every outbound HTTP call with latency histograms and error counters, capturing both success and failure paths.
• Label metrics by dependency name, method, status, and error type so you can alert and query with precision.
• Set alert rules on error rate and latency with appropriate for clauses (5m minimum) to avoid false positives.
• Build a dashboard showing dependency health by error rate, latency percentiles, and request volume.
• Establish naming conventions and baseline thresholds early so the team interprets metrics consistently.
• Treat monitoring third-party dependencies as a core reliability practice, not an afterthought—it's how you stay ahead of partner failures.