Testing across browsers: real-world lessons and examples

Introduction

Browser compatibility issues are expensive. A feature works perfectly in Chrome, ships to production, and suddenly your users on Safari experience a broken checkout flow or your Firefox users can't upload files. Testing across browsers: real-world lessons and examples shows us that the cost of catching these bugs early—during development—is a fraction of the cost of fixing them after they reach customers. In this article, we'll walk through how to structure a Playwright testing strategy that catches browser-specific bugs before they become production incidents.

We've all seen the browser compatibility nightmare. It usually happens like this: your team tests thoroughly on Chrome, everything passes, deployment happens, and within hours you're fielding support tickets from Safari users. The frustrating part is that the bug was often preventable with a simple cross-browser test.

Projects for Chromium, Firefox and WebKit

Playwright gives us three browser engines out of the box: Chromium (which powers Chrome and Edge), Firefox, and WebKit (which powers Safari). The smartest approach we've found is to define explicit projects in your playwright.config.ts that treat each browser as a distinct testing target. This forces your team to think about cross-browser compatibility from the start, not as an afterthought.

Here's how we structure this in practice:

import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  fullyParallel: true,
  forbidOnly: process.env.CI ? true : false,
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 1 : undefined,
  reporter: 'html',
  use: {
    baseURL: 'http://localhost:3000',
    trace: 'on-first-retry',
  },
  projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
    },
    {
      name: 'firefox',
      use: { ...devices['Desktop Firefox'] },
    },
    {
      name: 'webkit',
      use: { ...devices['Desktop Safari'] },
    },
  ],
  webServer: {
    command: 'npm run dev',
    url: 'http://localhost:3000',
    reuseExistingServer: !process.env.CI,
  },
});

Each project inherits the base configuration but can override specific settings. This structure means when you run npx playwright test, all three browsers execute the same test suite. The output clearly shows which tests pass or fail per browser, making it obvious when a bug is browser-specific.

We often add a fourth project for a mobile viewport using WebKit, since Safari on iOS is critically different from desktop Safari:

    {
      name: 'Mobile Safari',
      use: { ...devices['iPhone 12'] },
    },

This catches viewport-specific issues, touch interaction bugs, and iOS-specific CSS quirks that would otherwise slip through.

WebKit catches Safari issues early

Here's what we've learned: WebKit is your early warning system for Safari bugs. Roughly 90% of Safari-specific issues appear in WebKit tests before they hit real Safari users. This saves us from the nightmare of discovering bugs only after customers report them.

A concrete example: we had a payment form with custom input styling using CSS that worked perfectly in Chrome and Firefox. The input styling used a combination of appearance: none and gradient backgrounds. When we ran the suite against WebKit, it immediately failed—the input was rendering with the browser default styling, ignoring our CSS entirely. We investigated, found the issue was a specificity problem with how WebKit applies default styles, and fixed it before any Safari user ever saw it.

Another real scenario: our team built a file upload component using the Fetch API with custom progress tracking. It worked beautifully on Chromium and Firefox. WebKit tests revealed that Safari was silently failing on certain file types due to how it handles the FormData API with binary files. We caught this during development, not in production.

The key insight is treating WebKit as a mandatory gate, not an optional extra. If WebKit tests fail, the code doesn't merge. This shifts the mindset from "we'll handle Safari bugs when they happen" to "we prevent Safari bugs before they happen."

We've also found that WebKit catches subtle timing issues that Chrome overlooks. Playwright's WebKit engine is stricter about race conditions and async behavior, so tests that pass on Chrome but fail on WebKit often reveal genuine bugs hiding in your code.

Run all browsers nightly

Pull request builds are fast and tight—they need to be. But this creates a window where cross-browser regressions can slip through. The answer is a nightly job that runs your full test suite against all browsers, all devices, with retries and extensive logging. This catches issues that quick PR checks miss.

Here's our nightly workflow in GitHub Actions:

name: Nightly Cross-Browser Tests

on:
  schedule:
    - cron: '0 2 * * *'  # 2 AM UTC daily

jobs:
  test-all-browsers:
    runs-on: ubuntu-latest
    timeout-minutes: 60
    
    strategy:
      fail-fast: false
      matrix:
        browser: [chromium, firefox, webkit]
    
    steps:
      - uses: actions/checkout@v4
      
      - uses: actions/setup-node@v4
        with:
          node-version: '18'
      
      - name: Install dependencies
        run: npm ci && npx playwright install
      
      - name: Run tests on ${{ matrix.browser }}
        run: npx playwright test --project=${{ matrix.browser }}
      
      - name: Upload results
        if: always()
        uses: actions/upload-artifact@v3
        with:
          name: playwright-report-${{ matrix.browser }}
          path: playwright-report/
      
      - name: Notify on failure
        if: failure()
        run: |
          curl -X POST ${{ secrets.SLACK_WEBHOOK }} \
            -H 'Content-Type: application/json' \
            -d "{\"text\":\"Nightly ${{ matrix.browser }} tests failed\"}"

The critical elements here: run each browser in its own job to avoid cascading failures, use fail-fast: false to see all results, and artifact the HTML reports so you can inspect failures in detail. We also tag these nightly results with the commit hash so we can correlate failures to code changes.

The nightly suite is slower—we give it an hour—because it's more thorough. More retries, more trace collection, more device configurations. It catches flakiness and edge cases that tighter PR checks would skip. When a nightly run fails, we investigate immediately, even if PR tests passed.

Run one browser on every PR

Pull request builds need to stay fast. We can't run the full cross-browser suite on every PR—that's too slow, and developers won't wait. Instead, we run a subset on PRs and reserve the full suite for nightly.

Our strategy: run Chromium on every PR (as the baseline), Firefox once a day on a scheduled workflow, and WebKit in nightly only. This gives us quick feedback without creating a bottleneck.

name: PR Tests

on:
  pull_request:
    branches: [main, develop]

jobs:
  test-chromium:
    runs-on: ubuntu-latest
    timeout-minutes: 20
    
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '18'
      - run: npm ci && npx playwright install
      - run: npx playwright test --project=chromium
      
      - name: Comment PR on failure
        if: failure()
        uses: actions/github-script@v7
        with:
          script: |
            github.rest.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body: '❌ Chromium tests failed. Check the run for details.'
            })

Chromium typically completes in 5–8 minutes. It's fast enough that developers wait without frustration, yet comprehensive enough to catch obvious breakage. We then rely on nightly runs to catch Firefox and WebKit issues.

This tiered approach balances speed with safety. You get immediate feedback on PRs, comprehensive validation nightly, and Safari-specific checks that catch 90% of issues before they're deployed.

Common mistakes

• Running all browsers on every PR. It's thorough but unsustainable. Developers will disable or ignore slow CI. Fast feedback wins over perfect coverage every time.
• Treating browser-specific code as an afterthought. If you find yourself writing if (isSafari) hacks in your application code, you've already lost. Catch the issue in testing, fix the root cause in the code.
• Only testing happy paths. Cross-browser differences often show up in error states, edge cases, and async operations. Your tests need to exercise these scenarios.
• Ignoring WebKit failures because "it'll work in real Safari." It usually won't. WebKit is close enough to real Safari that failures are real problems, not false positives.
• Not tracking which browsers fail which tests over time. Without historical data, you can't spot patterns. A test that flakes only on Firefox once a week will keep burning developer time unless you track it.
• Mixing browser-specific logic in test files. Avoid test.skip('firefox', ...) or conditionals in test code. If a feature doesn't work on Firefox, fix the feature or document the limitation explicitly, don't hide the test.

Key takeaways

• Define explicit Playwright projects for Chromium, Firefox, and WebKit in your config. Treat each as a mandatory testing target, not optional validation.
• WebKit catches roughly 90% of real Safari issues during development. It's your early warning system—respect it.
• Run the full cross-browser suite nightly. Use separate jobs per browser, artifact reports, and alert on failure. Catch regressions before they're deployed.
• Run one browser (Chromium) on every PR for fast feedback. Reserve full cross-browser validation for scheduled nightly runs. Balance speed with safety.
• Browser-specific bugs are expensive in production. A 5–10 minute nightly test run prevents hours of firefighting. The ROI is obvious.
• Historical data matters. Track which tests fail on which browsers over time. Patterns will emerge that guide your development priorities.