Monitoring Nginx: a practical guide for developers

Introduction

Nginx sits at the edge of most of our infrastructure, yet we often treat its monitoring as an afterthought until a silent failure takes down production. Monitoring Nginx: a practical guide for developers means shifting from reactive firefighting to proactive visibility into proxy metrics and errors. This guide walks you through collecting request rates, connection states, upstream response times, and setting up alerts that actually fire when they should.

By the end of this article, you'll have a working setup that exposes Nginx metrics to Prometheus, visualizes them in Grafana, and alerts your team on 5xx spikes before your users notice.

Stub status or exporter modules

Before we can monitor anything, we need Nginx to expose metrics. You have two main paths: the built-in stub status module or a dedicated exporter like nginx-prometheus-exporter. The stub status approach is simpler but limited; the exporter gives you richer metrics with less wrestling with Nginx logs.

If you're building a new deployment, we recommend the exporter route. It's more battle-tested in production environments and handles reload gracefully. However, if you already have stub status enabled or want minimal dependencies, here's the stub status setup.

First, check if Nginx was compiled with --with-http_stub_status_module:

nginx -V 2>&1 | grep stub_status

If it's there, add a stub status endpoint to your Nginx config:

server {
  listen 127.0.0.1:8080;
  server_name localhost;

  location /nginx_status {
    stub_status on;
    access_log off;
    allow 127.0.0.1;
    deny all;
  }
}

Reload and test:

nginx -s reload
curl http://127.0.0.1:8080/nginx_status

You'll see output like:

Active connections: 42
server accepts handled requests
  1234 1234 5678
Reading: 2 Writing: 1 Waiting: 39

That's useful, but parsing text output in Prometheus is clunky. Instead, deploy nginx-prometheus-exporter alongside your Nginx instance. It scrapes stub status (or logs) and outputs Prometheus-format metrics. Download the binary or use Docker:

docker run -d \
  -p 4040:4040 \
  nginx/nginx-prometheus-exporter:0.11.0 \
  -nginx.scrape-uri=http://127.0.0.1:8080/nginx_status

Now curl the exporter:

curl http://127.0.0.1:4040/metrics | grep nginx_

You'll see metrics like nginx_connections_active, nginx_http_requests_total, and more. Add this to your Prometheus scrape config:

scrape_configs:
  - job_name: 'nginx'
    static_configs:
      - targets: ['127.0.0.1:4040']

Requests, connections and status codes

Once metrics are flowing into Prometheus, you can start building dashboards around the core signals: request throughput, connection counts, and HTTP status code distribution. These three metrics tell you most of what you need to know about Nginx health.

Request rate is your first alarm bell. In Grafana, create a panel with this query:

rate(nginx_http_requests_total[5m])

This shows requests per second over the last 5 minutes. A sudden drop usually means your upstream is down or hanging; a spike without a matching increase in errors might be organic traffic or a DDoS. Track it over time so you know what "normal" looks like for your service.

Connections tell a different story. Active connections, reading, writing, and waiting states help you understand if Nginx is saturated or if the problem is deeper. A high number of connections in the waiting state typically means your upstream is slow. Many in reading or writing could indicate network congestion or large payloads.

Create a stacked area chart:

nginx_connections_reading
nginx_connections_writing
nginx_connections_waiting

If waiting connections start climbing while your upstream stays fast, you have a traffic spike or a client issue. If reading or writing dominate, check upstream response times next.

Status codes are crucial. The exporter exposes nginx_http_requests_total with a status label, so you can break it down by 2xx, 4xx, 5xx, etc. A query like this shows you the error ratio:

sum(rate(nginx_http_requests_total{status=~"5.."}[5m])) / sum(rate(nginx_http_requests_total[5m]))

Even a small baseline of 5xx errors (from flaky upstreams or timeouts) is normal. Once you know your baseline, you can alert on meaningful deviations. We'll cover that in the alert section below.

A real example: one team running an e-commerce platform noticed request rates were normal but 4xx errors spiked after a deploy. Turns out a change to the proxy_pass path was breaking URLs. Catching that in Grafana saved them from relying on customer complaints to discover bugs.

Upstream response times from logs

Nginx stub status and the basic exporter only show you counts and connection states. To understand performance, you need upstream response times. This is where we shift from the exporter to log parsing.

Nginx logs request times in microseconds. Configure your log format to include the $upstream_response_time variable:

log_format main '$remote_addr - $remote_user [$time_local] "$request" '
                '$status $body_bytes_sent "$http_referer" '
                '"$http_user_agent" "$http_x_forwarded_for" '
                'upstream_response_time=$upstream_response_time '
                'request_time=$request_time';

access_log /var/log/nginx/access.log main;

Now a log line looks like:

192.168.1.1 - - [01/Jan/2025:12:34:56 +0000] "GET /api/users HTTP/1.1" 200 512 "-" "curl/7.68.0" upstream_response_time=0.125 request_time=0.130

To convert these into Prometheus metrics, use a log exporter like promtail (part of Loki) or the mtail tool. Here's an mtail program that parses Nginx logs:

# /etc/mtail/nginx.mtail
/upstream_response_time=(?P<upstream_time>[0-9.]+)/ {
  nginx_upstream_response_time_seconds.Observe(strptime(upstream_time, "%f"))
}

/status (?P<status_code>\d+)/ {
  nginx_http_requests_by_status.Inc(status_code)
}

Deploy mtail as a sidecar or systemd service, and it will stream metrics to a Prometheus text file or directly to a pull endpoint. Now you can query percentile latencies:

histogram_quantile(0.95, nginx_upstream_response_time_seconds_bucket)

This tells you the 95th percentile response time from your upstream. When this starts climbing, your app tier is struggling. Couple it with CPU and memory metrics on that tier, and you'll quickly spot the culprit.

In one SaaS team's case, they noticed p95 response times drifting from 50ms to 300ms over a week. Log-based metrics revealed that a new batch job was running during peak hours. Moving the job to off-peak immediately restored latency.

Alert on 5xx spikes

Having visibility is only half the battle; you need alerts that wake you up when Nginx is serving errors. Alerting on raw 5xx counts is noisy because traffic varies. Instead, alert on the ratio of 5xx to total requests, or compare it to a baseline.

In your Prometheus alerting rules file, add:

groups:
  - name: nginx_alerts
    interval: 30s
    rules:
      - alert: Nginx5xxSpikeDetected
        expr: |
          (sum(rate(nginx_http_requests_total{status=~"5.."}[5m])) 
          / sum(rate(nginx_http_requests_total[5m]))) > 0.05
        for: 2m
        labels:
          severity: warning
        annotations:
          summary: "Nginx 5xx error rate above 5%"
          description: "{{ $value | humanizePercentage }} of requests are 5xx errors on {{ $labels.instance }}"

      - alert: Nginx5xxCritical
        expr: |
          (sum(rate(nginx_http_requests_total{status=~"5.."}[5m])) 
          / sum(rate(nginx_http_requests_total[5m]))) > 0.10
        for: 30s
        labels:
          severity: critical
        annotations:
          summary: "Nginx 5xx error rate critical (>10%)"
          description: "{{ $value | humanizePercentage }} of requests are errors on {{ $labels.instance }}"

      - alert: NginxHighUpstreamLatency
        expr: |
          histogram_quantile(0.95, nginx_upstream_response_time_seconds_bucket) > 1
        for: 3m
        labels:
          severity: warning
        annotations:
          summary: "Upstream response time p95 > 1s"
          description: "p95 latency is {{ $value | humanizeDuration }}"

The thresholds (5%, 10%, 1 second) are tuning knobs. Start conservative and adjust based on what your team observes. The for: 2m clause prevents alert spam from transient blips.

Route these alerts to Slack, PagerDuty, or email. A quick win is to include a dashboard link in the annotation so on-call engineers jump straight to context:

annotations:
  dashboard: "https://grafana.example.com/d/nginx-overview?var-instance={{ $labels.instance }}"

One team we worked with had their alerting too sensitive, firing dozens of times a day. They added a 2–3 minute for clause and tuned the 5xx threshold to 3% (above their normal jitter but below actual incidents). Alert fatigue dropped to nearly zero, and they still caught all real outages.

Common mistakes

• Not baselining before alerting. Every service has a different error rate. Set a baseline over 1–2 weeks of normal operation before tuning alert thresholds. Otherwise, your alerts will either ignore real problems or cry wolf.
• Alerting on absolute request counts. A 5xx spike of 100 requests/sec might be normal for a high-traffic service but catastrophic for a small one. Always alert on rates or ratios relative to total traffic.
• Forgetting to account for upstream timeouts. When Nginx times out connecting to upstream, it logs the timeout but not always a clean 504 or 502. Check your error logs separately to catch these silent failures.
• Parsing logs without rate-limiting. If you run an mtail or log parser on high-traffic Nginx instances, it can become a bottleneck itself. Use buffering and ensure the tool doesn't block log writes.
• Mixing stub status metrics with exporter metrics. If you switch from stub status scraping to nginx-prometheus-exporter, metric names and labels change. Rewrite your Grafana queries and alerts, or you'll have blind spots.

Key takeaways

• Start with the nginx-prometheus-exporter for clean, reliable metrics; stub status is a fallback for minimal-dependency setups.
• Track requests, connections, and status codes as your core signals; alert on 5xx ratios, not absolute counts.
• Parse upstream response times from Nginx logs using mtail or promtail to catch slow upstreams before they cause cascading failures.
• Establish a baseline of error rates and latencies before setting alerts; tune thresholds to match your traffic patterns and risk tolerance.
• Link Grafana dashboards in alert annotations so your team can diagnose issues in seconds, not minutes.