SEO tests in CI: Technical SEO best practices explained

Search engine visibility is determined by more than rankings and backlinks—it lives in the technical implementation details your build pipeline can catch automatically. SEO tests in CI: Technical SEO best practices explained means asserting that your critical pages carry the right title tags, canonical URLs, metadata directives, and structured data before code ships to production. When you treat SEO as a build-time contract rather than a post-launch audit, you prevent the kind of technical regressions that silently erode search visibility across thousands of URLs.

Why SEO tests belong in CI

Teams that wait until quarterly SEO audits to discover broken canonicals or missing titles have already lost months of crawl budget and indexation. A misconfigured redirect, a missing robots meta tag, or a malformed JSON-LD schema might not crash your application, but it will crash your search visibility. By injecting SEO validation into your continuous integration pipeline, you shift left—catching these issues the moment a developer merges a change to a critical page template or route.

The mindset here mirrors any other quality gate: if a performance regression would fail your build, so should a technical SEO regression. This approach scales naturally across teams. A junior developer learns the SEO contract through CI feedback. A platform migration doesn't accidentally strip canonicals from thousands of pages because the test suite caught it first. A template refactor that removes a structured data block gets flagged before deployment.

Assert titles and canonicals on key routes

Page title tags and canonical URL elements are the foundation of technical SEO. The title tells search engines what your page is about; the canonical tells them which version is authoritative when duplicates exist. Both are simple HTML, yet both are frequently broken in build processes.

Start by identifying your critical routes—typically homepage, product/article listings, detail pages, and any high-traffic landing pages. For each, write a test that:

• Renders or crawls the page at build time
• Asserts a title tag exists, has length between 30–60 characters, and contains at least one keyword relevant to the page
• Asserts a canonical link element exists and points to the expected URL (usually self-referential for canonical pages)

Here's a practical example using Node.js and a headless browser approach:

// seo-tests.js
const puppeteer = require('puppeteer');

async function testSeoElements(url, expectedTitle, expectedCanonical) {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto(url, { waitUntil: 'networkidle2' });

  const title = await page.$eval('title', el => el.textContent);
  const canonical = await page.$eval(
    'link[rel="canonical"]',
    el => el.getAttribute('href')
  );

  if (!title || title.length < 30 || title.length > 60) {
    throw new Error(`Title invalid: "${title}"`);
  }
  if (canonical !== expectedCanonical) {
    throw new Error(`Canonical mismatch: got ${canonical}, expected ${expectedCanonical}`);
  }

  await browser.close();
  console.log(`✓ ${url} passed SEO assertions`);
}

// Run tests for critical routes
(async () => {
  await testSeoElements(
    'http://localhost:3000/products/widget-x',
    'Buy Widget X | Acme Store',
    'https://acme.com/products/widget-x'
  );
  await testSeoElements(
    'http://localhost:3000/blog/seo-tips',
    'Complete SEO Guide for Developers',
    'https://acme.com/blog/seo-tips'
  );
})();

This test runs during your build (or in a pre-deployment stage). If a developer accidentally removes a canonical tag or shortens a title to 20 characters, the build fails with a clear error message. No guessing, no surprises in Google Search Console three weeks later.

For Next.js and similar frameworks, you can extend this to test static pages at build time and validate against your route configuration automatically.

Check robots meta in production builds

The robots meta tag controls whether search engines can index and follow links on a page. A misconfigured robots="noindex" on your production homepage is a quiet catastrophe—the page remains live but becomes invisible to search engines.

Common scenarios where this breaks:

• A developer copies a staging environment build script that includes noindex globally and forgets to remove it for production
• A CMS update or feature flag accidentally applies noindex to product pages
• A middleware or layout component conditionally injects noindex based on environment, but the condition is inverted

Add a CI check that:

• Builds your site with production environment variables
• Checks that pages where noindex is unwanted do not carry that directive
• Allows noindex only on explicitly whitelisted routes (staging URLs, admin panels, etc.)

Example for a framework that generates static HTML:

// check-robots-meta.js
const fs = require('fs');
const path = require('path');
const glob = require('glob');

const NOINDEX_BLOCKLIST = [
  '/products/**/*.html',
  '/blog/**/*.html',
  '/index.html'
];

const NOINDEX_ALLOWLIST = [
  '/**/staging/**/*.html',
  '/**/admin/**/*.html'
];

function fileMatchesPattern(file, patterns) {
  return patterns.some(pattern => {
    const regex = new RegExp(pattern.replace(/\*\*/g, '.*').replace(/\*/g, '[^/]*'));
    return regex.test(file);
  });
}

function checkRobotsMeta() {
  const htmlFiles = glob.sync('dist/**/*.html');
  let failed = false;

  htmlFiles.forEach(file => {
    const content = fs.readFileSync(file, 'utf-8');
    const hasNoindex = content.includes('name="robots"') && content.includes('noindex');

    if (hasNoindex && fileMatchesPattern(file, NOINDEX_BLOCKLIST)) {
      console.error(`✗ ${file} has noindex but should be indexed`);
      failed = true;
    }

    if (!hasNoindex && fileMatchesPattern(file, NOINDEX_ALLOWLIST)) {
      console.error(`✗ ${file} should have noindex`);
      failed = true;
    }
  });

  if (failed) {
    process.exit(1);
  }
  console.log('✓ All robots meta tags are correct');
}

checkRobotsMeta();

Run this after your production build completes. It's fast, deterministic, and prevents the most common indexation disasters.

Validate JSON-LD in templates

Structured data (JSON-LD) helps search engines understand page content and can improve rich snippets in results. A malformed JSON-LD block isn't caught by your linter if it's dynamically generated, but it will be silently ignored by Google's parser.

Your test suite should:

• Render pages with sample data
• Extract JSON-LD blocks
• Validate against the relevant schema (Product, Article, Organization, etc.)
• Check for required fields and correct types

Use a library like jsonld or Google's Rich Results Test API in your test suite:

// validate-jsonld.js
const { toRdf } = require('jsonld');

async function validateProductSchema(html) {
  const scriptTag = html.match(/
  `;
  
  await validateProductSchema(testHtml);
})();

Integrate this validation into your template test suite. When a developer refactors a product card component and accidentally removes the offers.price field, the test catches it immediately rather than waiting for Google to re-crawl and notice the schema is incomplete.

Run Lighthouse CI on critical pages

Lighthouse isn't exclusively a performance tool—it includes SEO audits that check for common technical issues: mobile-friendliness, crawlable links, valid robots directives, and more. Lighthouse CI runs automated audits and can fail your build if scores drop below a threshold.

Set it up for your critical pages (homepage, key landing pages, product detail) so that:

• SEO regressions in accessibility, mobile experience, or structured data are caught
• Performance impacts on Core Web Vitals (which are ranking factors) are tracked
• Changes are compared against a baseline to spot unintended consequences

Example lighthouserc.json configuration:

{
  "ci": {
    "collect": {
      "url": [
        "http://localhost:3000",
        "http://localhost:3000/products",
        "http://localhost:3000/about"
      ],
      "numberOfRuns": 3,
      "settings": {
        "configPath": "./lighthouse-config.js"
      }
    },
    "upload": {
      "target": "temporary-public-storage"
    },
    "assert": {
      "preset": "lighthouse:recommended",
      "assertions": {
        "categories:seo": ["error", { "minScore": 0.90 }],
        "categories:accessibility": ["error", { "minScore": 0.85 }],
        "categories:performance": ["warn", { "minScore": 0.80 }]
      }
    }
  }
}

Add this to your CI workflow (GitHub Actions example):

- name: Run Lighthouse CI
  uses: treosh/lighthouse-ci-action@v10
  with:
    configPath: './lighthouserc.json'
    runs: 3
    temporaryPublicStorage: true

An SEO score below 0.90 fails the build, catching issues like missing alt text, broken links, or poor mobile rendering before they reach production.

Common mistakes

Testing only staging, not production builds. Staging environments often have different environment variables, middleware, or CMS settings than production. Always run your SEO tests against a production build to catch environment-specific bugs.

Ignoring dynamic routes and server-rendered content. If your app uses dynamic routing or server-side rendering, hardcoding a test for /products/widget-x won't help you catch issues on /products/widget-y. Use parameterized tests or crawl sample data across route patterns.

Setting thresholds too loose. A Lighthouse SEO score of 0.70 is not acceptable for a public website. Set meaningful thresholds (0.90+) that require actual adherence to SEO best practices, not just "not completely broken."

Not versioning schema definitions. JSON-LD schemas evolve. If you hardcode a validation rule for a Product schema and Google later recommends an additional optional field, your test might become outdated. Use schema libraries that stay current with spec changes.

Skipping non-critical pages. You don't need to test every page, but do test representative samples—category pages, blog articles, edge cases like 404 pages. A 404 with noindex is correct; a 404 with an identical title tag as your homepage is not.

Key takeaways

• Treat technical SEO as a build-time contract: title length, canonical presence, robots directives, and structured data should fail the build if misconfigured.
• Assert titles and canonicals on key routes using headless browser testing or static HTML inspection, ensuring they match expected values before deployment.
• Check robots meta tags against blocklists and allowlists to prevent accidental noindex on production content or missing noindex on staging URLs.
• Validate JSON-LD schemas in templates by parsing and checking for required fields and correct types, catching malformed structured data early.
• Run Lighthouse CI on critical pages to detect SEO regressions in mobile-friendliness, accessibility, and Core Web Vitals alongside performance.
• Run these tests against production builds and representative samples of your route patterns, not just hardcoded URLs or staging environments.
• Set meaningful score thresholds (0.90+ for SEO and accessibility) that require genuine compliance, not just token adherence.