Crawled is not indexed: a practical guide for developers

You've built a new feature, deployed it to production, and submitted the URL to Google Search Console. Two weeks later, you check indexation and see "Crawled — currently not indexed" next to your page. Your first instinct is to panic and hammer the re-index button. But here's the truth: that status isn't a bug or a temporary glitch. It's Google telling you something concrete about your page's value, and ignoring it will waste your time.

In this post, we'll decode what that status really means, why it happens, and most importantly, how to fix it without spam-hammering the search console.

Google fetched the page but chose not to keep it

Let me be direct: when Google says "crawled — currently not indexed," it means Google's crawler successfully downloaded your page, parsed it, understood what it was about, and then made a deliberate choice not to store it in the search index.

This isn't a crawl error. Your server didn't reject Google. The page is technically live and accessible. Google just decided the page wasn't worth keeping in its index for serving to users.

Think of Google's index like a library. The librarian (Googlebot) can walk into every room and read every book on the shelf. But at the end of the day, the library only keeps books on its main shelves if they're valuable to library members. If a book is too similar to one already on the shelf, or if it's so thin it barely has any substance, the librarian puts it in storage rather than shelving it prominently.

This is where many developers get confused. They assume indexation is binary: either the page is indexed or it's blocked by robots.txt or a noindex tag. But Google maintains a middle ground. The page is in Google's systems. Google knows it exists. But it's not in the active search index where users can find it.

From a developer's standpoint, this usually surfaces when:

• You check Google Search Console and filter by indexation status
• You run a site: search and notice certain pages don't appear
• You use Google's URL inspection tool and see this specific status

The critical insight: this is a signal from Google about page quality and relevance, not a technical problem to "fix" by resubmitting.

Often thin, duplicate or low-value content

Now that we know Google made a conscious choice, let's talk about why. The most common reasons a page lands in "crawled but not indexed" are:

Thin or low-value content

Your page has minimal unique value. It might be under 300 words, rehash the same information as your homepage, or provide no context that a user would actually need.

Real example: You built a CRM platform. For each client in your system, you auto-generated a public profile page with just the company name, logo, and a one-line description. No additional data, no unique insight, no reason someone would search for it directly. Google crawled it, recognized it as thin, and didn't index it.

Duplicate or near-duplicate content

This is the second most common culprit. You have multiple URLs serving identical or nearly identical content without proper canonicalization or URL parameters are creating accidental duplicates.

For example, pagination URLs like /products?page=1, /products?page=2, or session-based URLs like /checkout?sid=abc123 might all render similar content. Google crawls them, understands they're redundant, and only indexes the canonical version.

Soft 404s and redirect chains

Your page returns a 200 status code, but the content is minimal or a placeholder. The server isn't explicitly saying "this page doesn't exist," but the page itself signals "nothing valuable here." Google respects that signal.

Noindex or robots exclusion mistakes

Less common but worth checking: you might have accidentally set noindex in the page header, or your robots.txt is blocking crawl access to assets that impact rendering quality. Google can still fetch and parse the page, but it won't index it.

No inbound links or poor internal linking

If a page has no internal links pointing to it and no external backlinks, Google has no reason to prioritize it for indexation. It's essentially isolated in your site structure.

When I was building a trading platform, we created a help center with 200+ articles. Pages that were buried three levels deep with no inbound links consistently ended up in "crawled but not indexed." Pages that were linked from the main navigation or from other high-authority pages on the site? Indexed within days.

Improve the page or consolidate it

Here's your action plan. You have two legitimate paths forward.

Option 1: Improve and enrich the page

If the page has value but is thin, beef it up.

• Add substantive content. Expand to at least 500–800 words if it's a blog post or help article. Provide real depth. Don't pad with fluff, but give users and Google reasons to keep the page.
• Add internal links strategically. Link to it from related pages and from your main navigation or hub pages. Make it part of your site's structure, not an orphan.
• Improve on-page signals. Use a clear H1, properly structured headings, meta description, and schema markup where appropriate. Make it easy for Google to understand what the page is about.
• Add unique data or media. If it's product-related, include actual specs, images, or comparisons. If it's informational, include examples, case studies, or original research.

Example: You have a blog post titled "How to use our REST API." It's 250 words of generic explanation. Improve it by:

• Adding a full code walkthrough with authentication, pagination, and error handling
• Including a real-world use case (e.g., "Building a webhook receiver")
• Linking to related posts on SDKs, webhooks, and rate limiting
• Adding a table comparing REST vs. GraphQL endpoints

After two weeks, check Google Search Console again. Often, simply enriching the page will get it indexed, because you've now given Google substantive, unique content to rank.

Option 2: Consolidate into a stronger page

If the page is inherently redundant or low-value, kill it and consolidate.

Set up a 301 redirect from the weak page to a stronger, related page. Update your internal links to point to the consolidated URL instead.

Why this works: Instead of Google managing two mediocre pages, you now have one solid page with more inbound links, more content, and clearer relevance. This concentrated authority is easier to index and rank.

Practical example: You have a SaaS product with both a /features/reporting page and a /reporting page that cover nearly the same thing. The /reporting page is thin and not indexed. Redirect it to /features/reporting, which becomes richer and now has link equity from both URLs.

When to use each strategy

Ask yourself: "Is this page unique and strategically important?" If yes, improve it. If no, consolidate it. Don't keep pages around just because they exist.

Do not spam re-index requests

This is where I see developers and marketers make their biggest mistake.

When you see "crawled but not indexed," your instinct might be to:

• Use the "Request indexing" button in Google Search Console obsessively
• Resubmit your sitemap every day
• Make tiny changes to the page (adding a space, removing a space) and resubmit
• Ping Google via every possible mechanism

Stop. This doesn't work, and it signals spam-like behavior.

Here's why: Google already knows about the page. It crawled it. The reason it's not indexed isn't a discovery problem or a "Google missed it" problem. It's a quality signal. Spamming re-index requests doesn't change the underlying quality issues. It just adds noise to Google's systems and potentially trains Google to deprioritize re-crawls of your site.

Google's guidance is explicit: use the re-index request tool only once per URL, after you've made meaningful improvements. Not before. Not every day.

What you should do instead:

1. Make real changes to the page. Add content, improve structure, fix technical issues, enrich with schema markup.
2. Wait 1–2 weeks. Google will re-crawl your pages regularly. You don't need to force it.
3. If the page still isn't indexed, use the re-index button once. Then wait another 1–2 weeks.
4. Check indexation status in Search Console. You'll see updates in the "Coverage" report.

The most reliable signal to Google that a page is worth indexing is earning inbound links and getting organic clicks from search. If users search for your content and click through to the page, Google sees demand. That's a stronger signal than any resubmit request.

Common mistakes

• Assuming it's a technical bug. It's not. The page is fetched correctly. The issue is quality or redundancy, not infrastructure.
• Adding noindex by accident. Double-check your page headers and template logic. A stray noindex directive on hundreds of pages will tank indexation. I've seen this in Next.js projects where middleware was incorrectly adding noindex to all dynamic routes.
• Ignoring the Content Analysis report in Search Console. It shows you exactly which pages are marked as duplicate or low-value. Read it. Act on it.
• Creating pages with no internal linking strategy. Orphaned pages stay orphaned. Always link to new pages from at least one hub or navigation element.
• Confusing "crawled but not indexed" with "excluded by robots.txt." These are different statuses in Search Console. Know the difference. If it says "crawled but not indexed," the page is reachable; Google just chose not to keep it.

Key takeaways

• "Crawled — currently not indexed" is a quality signal, not an error. Google successfully fetched your page and decided it wasn't valuable enough for the search index. Respect that signal.
• The most common causes are thin content, duplicates, or poor internal linking. Audit your pages. Enrich thin ones, consolidate duplicates, and link strategically.
• Fix the root issue before resubmitting. Make real improvements to content, structure, or technical signals. Then use the re-index request exactly once. Spamming requests doesn't help.
• Consolidation is often better than creation. Instead of maintaining multiple mediocre pages, combine them into one authoritative page with more content and more inbound links.
• Internal linking and organic demand matter more than resubmit buttons. Link to your pages from high-authority sections of your site. Earn clicks from users. Google will index what serves users best.