Synthetic monitoring with Playwright: a practical guide

Synthetic monitoring with Playwright: a practical guide

Your API says it's up, your database shows green, yet customers report that checkout fails silently. You miss it for forty minutes because no one was actively using the flow. Synthetic monitoring fills this gap: you run your critical user journeys automatically on a schedule, catch real problems before customers do, and get alerted when something breaks.

In this guide, we'll walk through building and deploying synthetic monitoring with Playwright, the same tool many of us use for end-to-end testing. We'll cover running critical journeys on a schedule, integrating with platforms like Checkly, setting up intelligent alerting, and tracking response times that matter to your users.

Run critical journeys on a schedule

Synthetic monitoring isn't about testing everything. It's about testing the paths your users rely on most—login, adding to cart, payment processing, account recovery. You run these flows periodically, often every 5–15 minutes from multiple geographic regions, and measure what actually happens.

Start by identifying your critical journeys. For an e-commerce platform, that might be:

• User login and session establishment
• Product search and filtering
• Add to cart and view cart
• Checkout and payment confirmation
• Post-purchase email delivery verification

Each journey is a Playwright script that mimics a real user. Here's a minimal example for a login + product view flow:

import { test, expect } from '@playwright/test';

test('critical: login and view product', async ({ page, context }) => {
  const startTime = Date.now();
  
  // Navigate to login
  await page.goto('https://app.example.com/login', { waitUntil: 'networkidle' });
  expect(page).toHaveTitle(/Login/);
  
  // Login flow
  await page.fill('input[name="email"]', 'monitor@example.com');
  await page.fill('input[name="password"]', process.env.MONITOR_PASSWORD);
  await page.click('button[type="submit"]');
  
  // Wait for redirect and dashboard load
  await page.waitForURL('**/dashboard', { timeout: 10000 });
  await expect(page.locator('[data-testid="welcome-message"]')).toBeVisible();
  
  // Navigate to a product
  await page.goto('https://app.example.com/products/featured');
  await expect(page.locator('[data-testid="product-list"]')).toBeVisible({ timeout: 8000 });
  
  const duration = Date.now() - startTime;
  console.log(`Journey completed in ${duration}ms`);
});

Key elements here: we measure total time, we wait for meaningful states (not just DOM ready), and we use explicit data attributes for stability. Timeouts are tight because we want to catch slowness, not wait indefinitely.

For scheduling, most organizations run these checks every 5–10 minutes during business hours, and less frequently (or not at all) during maintenance windows. You'll also want to run them from multiple regions—US East, Europe, Asia—to catch regional failures early.

Checkly and similar tools run Playwright scripts

You could run Playwright scripts yourself on a cron job in your infrastructure, but dedicated synthetic monitoring platforms make this simpler. Checkly, Grafana k6, and DataDog RUM all offer Playwright integration. They handle scheduling, result storage, alerting, and multi-region execution without you managing containers or servers.

Checkly's model is straightforward: write a Playwright test, upload it, configure the schedule and regions, and Checkly runs it for you. Here's what a Checkly setup looks like in code (their config file):

import { defineConfig } from 'checkly/constructs';
import { Frequency } from 'checkly/constructs';

const config = defineConfig({
  projectName: 'E-commerce Monitoring',
  logicalId: 'ecommerce-prod-monitoring',
  repoUrl: 'https://github.com/yourorg/monitoring-scripts',
  checks: [
    {
      name: 'Login and Product View',
      type: 'browser',
      frequency: Frequency.EVERY_5_MINUTES,
      locations: ['us-east-1', 'eu-west-1', 'ap-southeast-1'],
      script: './checks/login-product-view.spec.ts',
      timeout: 30000,
      alertChannels: ['slack-prod', 'pagerduty'],
    },
    {
      name: 'Checkout Flow',
      type: 'browser',
      frequency: Frequency.EVERY_10_MINUTES,
      locations: ['us-east-1', 'eu-west-1'],
      script: './checks/checkout.spec.ts',
      timeout: 45000,
      alertChannels: ['slack-prod', 'pagerduty'],
    },
  ],
});

export default config;

The platform handles the infrastructure. Your scripts stay in version control, and you update them like any other code. Checkly runs them from their own browsers in multiple regions, so you get a true external view of your system—not affected by your internal network or CDN policies.

If you prefer to self-host, you can run Playwright scripts on a schedule using GitHub Actions, GitLab CI, or a simple container on Kubernetes with a CronJob. The trade-off is you maintain scheduling, storage, and alerting yourself, but you gain full control and can run checks from your own regions or infrastructure.

Alert on failures

A check that fails silently is useless. Set up alerts that go to your team immediately—Slack, PagerDuty, email, whatever your incident response workflow uses.

Smart alerting includes:

• Multi-region correlation: If a check fails in one region, you might just ignore it (local network blip). If it fails in three regions simultaneously, that's a real incident.
• Flakiness filtering: Retry once automatically before alerting. Many transient failures clear on retry.
• Escalation policies: First failure might go to Slack. After five consecutive failures over 15 minutes, page the on-call engineer.
• Metadata enrichment: Include the journey name, which region failed, the error message, and a link to the full run log.

Here's what an alert payload might look like in Slack:

🚨 Synthetic Monitor Alert
Journey: Login and Product View
Status: Failed
Regions: us-east-1, eu-west-1, ap-southeast-1
Error: Timeout waiting for dashboard after login (>10s)
First failure: 2024-01-15 14:32:15 UTC
Consecutive failures: 2
Dashboard: [View Run Details](https://checkly.com/runs/xyz)

Most platforms support conditional alerting through their API. For example, with Checkly, you can add custom logic:

import { test, expect } from '@playwright/test';

test('checkout flow with alerting logic', async ({ page }, testInfo) => {
  const startTime = Date.now();
  
  try {
    await page.goto('https://app.example.com/checkout');
    await page.fill('input[name="card"]', '4111111111111111');
    await page.click('button:has-text("Complete Purchase")');
    
    await expect(page.locator('[data-testid="success-message"]')).toBeVisible({ timeout: 15000 });
  } catch (error) {
    // Attach debugging info to the run
    testInfo.attach('screenshot-on-failure', {
      body: await page.screenshot(),
      contentType: 'image/png',
    });
    
    // Re-throw to fail the check
    throw error;
  }
  
  const duration = Date.now() - startTime;
  testInfo.attach('performance-metrics', {
    body: JSON.stringify({ checkoutDurationMs: duration }),
    contentType: 'application/json',
  });
});

Include screenshots and logs with failures. When the on-call engineer gets paged at 3 AM, a full screenshot of what the user saw is worth its weight in gold.

Track response times

Beyond pass/fail, track how fast your critical journeys complete. Slow checkout isn't a failure—it's a degradation. Response time metrics let you catch performance regressions before they become outages.

Extract and report metrics from your Playwright scripts:

import { test, expect } from '@playwright/test';

interface JourneyMetrics {
  timeToLogin: number;
  timeToProductList: number;
  timeToAddCart: number;
  totalJourneyTime: number;
}

test('track response times: add to cart flow', async ({ page }, testInfo) => {
  const metrics: JourneyMetrics = {
    timeToLogin: 0,
    timeToProductList: 0,
    timeToAddCart: 0,
    totalJourneyTime: 0,
  };
  
  const journeyStart = Date.now();
  
  // Login
  const loginStart = Date.now();
  await page.goto('https://app.example.com/login');
  await page.fill('input[name="email"]', 'monitor@example.com');
  await page.fill('input[name="password"]', process.env.MONITOR_PASSWORD);
  await page.click('button[type="submit"]');
  await page.waitForURL('**/dashboard');
  metrics.timeToLogin = Date.now() - loginStart;
  
  // Browse products
  const browseStart = Date.now();
  await page.goto('https://app.example.com/products?category=featured');
  await expect(page.locator('[data-testid="product-grid"]')).toBeVisible();
  metrics.timeToProductList = Date.now() - browseStart;
  
  // Add to cart
  const cartStart = Date.now();
  await page.click('[data-testid="add-to-cart-btn"]:first-of-type');
  await expect(page.locator('[data-testid="cart-notification"]')).toBeVisible();
  metrics.timeToAddCart = Date.now() - cartStart;
  
  metrics.totalJourneyTime = Date.now() - journeyStart;
  
  // Attach metrics for Checkly to ingest
  testInfo.attach('metrics', {
    body: JSON.stringify(metrics),
    contentType: 'application/json',
  });
  
  // Log for visibility
  console.log(`Login: ${metrics.timeToLogin}ms, Browse: ${metrics.timeToProductList}ms, Cart: ${metrics.timeToAddCart}ms`);
});

Most platforms (Checkly, DataDog, New Relic) automatically parse these metrics and plot them over time. Set thresholds—alert if login takes >5 seconds, checkout >15 seconds—and you'll catch performance issues as they emerge, not after customers complain.

Correlate synthetic response times with real user monitoring (RUM) data. If synthetic times are normal but RUM shows slowness, the problem is in the client or specific user network conditions, not your backend.

Common mistakes

Brittle selectors: Using CSS selectors that change with every design refresh. Commit to data-testid attributes early and maintain them religiously.

Ignoring maintenance windows: Your monitoring script hits checkout payment processing during a scheduled database migration. Use feature flags or skip-flags in your check configuration to disable specific journeys during known maintenance.

Monitoring the wrong thing: Setting up synthetic checks for flows no one uses, while missing the real conversion path. Analyze your actual user behavior first.

Alert fatigue: Setting thresholds so tight that every network hiccup triggers an alert. Start conservative (e.g., 2+ consecutive failures, 30+ second timeout), refine as you see patterns.

Using production data without isolation: Your monitoring account creates real orders or shipments. Use a dedicated monitoring user, test environment, or stub sensitive operations with request interception.

No retry logic: Flaky networks cause transient failures. Implement automatic retry with backoff before alerting. One failure isn't an incident; five consecutive failures in five minutes is.

Key takeaways

• Synthetic monitoring runs your critical user journeys on a schedule to catch real problems before customers do.
• Platforms like Checkly run Playwright scripts from multiple regions automatically, removing the operational burden of scheduling and storage.
• Alert on failures with multi-region correlation, retry logic, and rich context (screenshots, logs, metadata) so your team can respond fast.
• Track response times within each journey segment to catch performance regressions early, not after they degrade revenue.
• Use data-testid selectors, isolate test data, and set realistic thresholds to avoid brittle tests and alert fatigue.
• Correlate synthetic metrics with real user monitoring to distinguish between real outages and client-side problems.