Infinite scroll that Google can crawl: a practical guide

Infinite scroll that Google can crawl

Users love infinite scroll. Crawlers need pages. This tension has cost plenty of teams months of lost organic traffic because they built beautiful, fluid experiences that search engines couldn't follow. I've seen it happen on e-commerce platforms, news sites, and SaaS dashboards—the infinite scroll works perfectly on desktop, engagement metrics look great, but Google indexes only the first batch of items. The good news: infinite scroll and SEO aren't mutually exclusive. You can deliver both a modern user experience and a crawlable architecture by backing infinite scroll with paginated URLs.

Why infinite scroll breaks SEO by default

Traditional infinite scroll loads new content dynamically as users scroll down, but the URL never changes. From Google's perspective, every user is on the same page. If you have 10,000 products and they all load via JavaScript on /products, Google sees only one page with whatever content loaded in its initial render pass. The crawler doesn't scroll, doesn't trigger your intersection observer, doesn't fetch the next batch.

This isn't malice—it's a fundamental mismatch. Crawlers are stateless. They request a URL, parse the response, extract links, and move on. They don't maintain a scroll position or execute JavaScript interactions indefinitely. Pagination, by contrast, is built into how crawlers work: each page has its own URL, its own entry point, and explicit next/previous links.

The solution isn't to abandon infinite scroll. It's to implement it on top of a paginated foundation that crawlers can understand.

Back infinite scroll with paginated URLs

Every batch of content your infinite scroll loads should correspond to a real, crawlable page. If you're loading 20 products per batch, /products?page=1 should serve 20 items, /products?page=2 should serve the next 20, and so on.

On the server side, this is straightforward. Your API or server-rendered template already knows how to paginate:

// Next.js API route (pages/api/products.js)
export default async function handler(req, res) {
  const page = parseInt(req.query.page) || 1;
  const limit = 20;
  const skip = (page - 1) * limit;

  const products = await db.products
    .find()
    .skip(skip)
    .limit(limit)
    .lean();

  const total = await db.products.countDocuments();
  const hasMore = skip + limit < total;

  res.json({
    data: products,
    page,
    hasMore,
    total
  });
}

On the client, infinite scroll still works exactly as your users expect. When they scroll near the bottom, fetch the next page's data and append it to the DOM. The crawler, meanwhile, can visit /products?page=1, /products?page=2, and so on independently.

This architecture also keeps your CMS or backend simple. You're not building separate "infinite scroll endpoints" and "pagination endpoints"—you have one source of truth that serves both use cases.

Update the URL as users scroll

Backing content with paginated URLs is essential, but it only works if users can actually reach those pages. The next step is updating the browser's URL as users scroll. When someone scrolls down and you load page 2, the URL should become /products?page=2. This way:

• Users can bookmark or share the URL and land on the correct scroll position.
• The browser history works as expected—back button goes to the previous page.
• Crawlers following links in your site navigation land on pages with actual content, not a blank slate.

Use the History API to update the URL without reloading:

// React example with Next.js
import { useEffect, useRef, useCallback } from 'react';
import { useRouter } from 'next/router';

export default function ProductList() {
  const router = useRouter();
  const observerTarget = useRef(null);
  const currentPage = parseInt(router.query.page) || 1;
  const [items, setItems] = useState([]);
  const [hasMore, setHasMore] = useState(true);

  const loadMore = useCallback(async () => {
    const nextPage = currentPage + 1;
    const res = await fetch(`/api/products?page=${nextPage}`);
    const json = await res.json();

    setItems(prev => [...prev, ...json.data]);
    setHasMore(json.hasMore);

    // Update URL without full reload
    window.history.pushState(
      { page: nextPage },
      '',
      `/products?page=${nextPage}`
    );
    router.replace(`/products?page=${nextPage}`, undefined, { shallow: true });
  }, [currentPage, router]);

  useEffect(() => {
    const observer = new IntersectionObserver(
      entries => {
        if (entries[0].isIntersecting && hasMore) {
          loadMore();
        }
      },
      { threshold: 0.1 }
    );

    if (observerTarget.current) {
      observer.observe(observerTarget.current);
    }

    return () => observer.disconnect();
  }, [loadMore, hasMore]);

  return (
    <div>
      {items.map(item => (
        <div key={item.id}>{item.name}</div>
      ))}
      <div ref={observerTarget}>Loading...</div>
    </div>
  );
}

The key is window.history.pushState(). It updates the URL and browser history without a page reload. Users stay in flow, and the address bar reflects their actual position.

Provide links to each page

JavaScript can fail, networks can be slow, and some users disable JavaScript entirely. More importantly, crawlers don't run JavaScript by default. You need explicit HTML links to every page so that crawlers can discover and follow them.

Add a pagination nav section at the bottom of your content, or in a footer. Include rel=next and rel=prev tags in the <head> to signal pagination structure to search engines:

<!-- In the <head> of your page -->
<link rel="next" href="/products?page=2" />
<link rel="prev" href="/products?page=1" />

<!-- In the page footer or below content -->
<nav aria-label="Pagination">
  <ul>
    <li>
      <a href="/products?page=1">1</a>
    </li>
    <li>
      <a href="/products?page=2">2</a>
    </li>
    <li>
      <a href="/products?page=3">3</a>
    </li>
    <li><a href="/products?page=4">Next</a></li>
  </ul>
</nav>

These links serve a dual purpose: they're fallback navigation for users on slow connections or with JavaScript disabled, and they're explicit crawlable paths for search engines. Google will follow them and index each page independently.

If your dataset is very large (thousands of pages), consider using a sitemap to list all paginated URLs instead of cluttering the UI. But always include at least next/previous links on the page itself.

Test with JavaScript disabled

The ultimate test of whether your infinite scroll is truly crawlable is turning off JavaScript and seeing if the site still works. Open Chrome DevTools, press Ctrl+Shift+P (or Cmd+Shift+P on Mac), search for "Disable JavaScript," and reload the page.

What you should see:

• The first page of content loads and displays fully.
• Pagination links are visible and clickable.
• Clicking "Next" takes you to /products?page=2 with a full page load, and page 2's content displays.
• The browser back button works correctly.

What you shouldn't see:

• A blank page with a loading spinner that never resolves.
• Hidden pagination links that only appear after JavaScript runs.
• All content crammed into one URL.

If your infinite scroll passes the "no JavaScript" test, it will pass Google's crawl test. This is because Google's rendering engine is much more capable than a browser with JavaScript disabled, but it still has limitations on DOM interactions and scroll events.

I also recommend using Google's URL Inspection tool in Search Console to see exactly what Google renders when it crawls your page. Request indexing of a paginated URL and inspect it—you'll see whether the links are visible to the crawler and which content was captured.

Common mistakes

Loading all data client-side, then paginating in JavaScript: Some teams fetch 10,000 products in one initial request and paginate them with JavaScript. This defeats the purpose: the crawler sees the entire payload and doesn't understand the pagination boundaries. Pagination must happen on the server.

Hiding pagination behind JavaScript components: Pagination links that only render after hydration are invisible to crawlers. Always render pagination links as part of the initial HTML, even if infinite scroll is the primary UI.

Using hash-based pagination (#page=2): Hash-based URLs are treated as a single page by crawlers and are harder to track in analytics. Use query parameters instead.

Forgetting to set canonical tags: On paginated content, each page should have a self-referential canonical tag (<link rel="canonical" href="/products?page=2" />) to avoid duplicate content issues.

Not testing crawlability: Many teams assume their infinite scroll is crawlable without actually verifying it. Always test with JavaScript disabled and use Google's Inspect tool.

Key takeaways

• Infinite scroll and SEO coexist when you back infinite scroll with server-side paginated URLs. Each batch of content should correspond to a real page with its own URL.
• Update the browser URL as users scroll using the History API so pagination is visible in the address bar and users can share/bookmark.
• Provide explicit HTML pagination links (with rel=next and rel=prev) so crawlers can discover all pages independently, even without JavaScript.
• Test your implementation with JavaScript disabled in the browser. If pagination and content still work, Google will crawl it.
• Use Google Search Console's URL Inspection tool to verify that crawlers see your paginated content as separate pages, not as a single page with dynamic content.