Common search relevance mistakes: lessons from production

Introduction

After months in production, your search feature starts getting complaints. Users say results feel wrong. Businesses lose conversions because customers can't find what they're looking for. The problem isn't usually Elasticsearch itself—it's how we configure it. We've seen this pattern repeatedly across e-commerce platforms, SaaS dashboards, and travel services: teams launch search, iterate on features, then discover their relevance tuning was fundamentally broken from day one.

Common search relevance mistakes: lessons from production reveals the core issues we encounter when auditing search implementations. These aren't exotic edge cases. They're mistakes that happen because relevance feels like a black box, and the pressure to ship outweighs the discipline to measure. Let's walk through what goes wrong and how to fix it.

No field boosts

By default, Elasticsearch treats all fields equally. If a product query matches both the product name and a description paragraph, both scores count. But in reality, a match in the product title should rank much higher than a match buried in a long description. Without field boosts, your results feel random.

We worked with an e-commerce platform struggling with search. Their query for "blue shoes" returned results where "blue" appeared once in the title but "shoes" appeared fifteen times in the description—and the description match ranked higher. Field boosts fix this immediately.

Here's a basic multi-field search with boosts applied:

{
  "query": {
    "multi_match": {
      "query": "blue shoes",
      "fields": [
        "name^5",
        "sku^3",
        "description^1",
        "category^2"
      ],
      "type": "best_fields"
    }
  }
}

In this example, matches in the name field are worth 5x more than matches in the description. The sku gets a 3x boost because SKU matches are often exact or near-exact. The category gets 2x weight because it's structured data.

The boost values aren't magic. They're learned through iteration and metrics. Start conservative—1x for low-priority fields, 3-5x for high-priority ones. Then measure how rankings change and adjust. A 5x boost on brand name in a fashion vertical might be too aggressive; a 2x boost might be right. You won't know without testing.

Common beginner mistake: setting boosts without understanding what you're optimizing for. Is an exact SKU match more important than a category match? That depends on your business. Before you set any boost value, clarify what users actually want to find.

Ignoring synonyms

Language is inconsistent. Users search for "couch" but your catalog has "sofa." A user searches "sneaker" but you indexed "athletic shoe." Without synonym configuration, these queries fail or return irrelevant results.

Synonyms aren't just about variant names—they're about intent. In a travel platform, "departure" and "outbound" should be synonymous. In a medical SaaS, "hypertension" and "high blood pressure" must map to the same results. The business cost of missing this is high.

Set up synonyms in your analyzer configuration, not your mapping:

{
  "settings": {
    "analysis": {
      "filter": {
        "synonym_filter": {
          "type": "synonym",
          "synonyms": [
            "couch,sofa,settee",
            "sneaker,athletic shoe,running shoe",
            "vehicle,car,automobile",
            "departure=>outbound",
            "hypertension=>high blood pressure"
          ]
        }
      },
      "analyzer": {
        "search_analyzer": {
          "tokenizer": "standard",
          "filter": ["lowercase", "synonym_filter"]
        }
      }
    }
  }
}

Two strategies here. Bidirectional synonyms (couch,sofa,settee) treat all terms as equivalent. Directional mappings (hypertension=>high blood pressure) normalize to a canonical form. Use directional mapping when one term is authoritative.

The hard part isn't configuration—it's maintaining synonyms. A spreadsheet with 50 synonym pairs is manageable. A spreadsheet with 5,000 is chaos. Invest in tooling to manage and version synonyms. Log searches that yield no results; those are candidates for new synonym pairs. Update synonyms quarterly based on user behavior and catalog changes.

One team we worked with ignored synonyms entirely, then wondered why conversion dropped when they changed product names. A simple synonym audit would have caught this.

Fuzzy everything

Fuzzy matching lets you find "shose" when searching for "shoes." It's powerful. It's also seductive, and teams often apply it too broadly.

Fuzzy matching is computationally expensive. It expands the query space dramatically, slowing down searches. Worse, if you apply it to every field at high edit distances, you get noise. A fuzzy match for "blue" might return "glue" or "blur"—technically correct, but useless to your user.

Here's an aggressive (and usually wrong) approach:

{
  "query": {
    "match": {
      "description": {
        "query": "shose",
        "fuzziness": "AUTO"
      }
    }
  }
}

This will find "shoes," "shone," "shore," and other variations. In some contexts that's right. But if you apply fuzziness: AUTO across all fields, you'll notice:

• Queries get slow (especially on large datasets)
• Results include false positives
• Users don't understand why they got a certain result

Better approach: apply fuzzy matching only where it matters. For product names and SKUs, use a low fuzziness (1 edit distance). For descriptions and long text, skip fuzzy entirely. For exact matches like order IDs, never use fuzzy.

{
  "query": {
    "bool": {
      "should": [
        {
          "match": {
            "name": {
              "query": "shose",
              "fuzziness": 1,
              "boost": 10
            }
          }
        },
        {
          "match": {
            "description": {
              "query": "shose"
            }
          }
        }
      ]
    }
  }
}

Here, a fuzzy match on the name gets high boost and low fuzziness. A non-fuzzy match on description is a fallback. This limits computational overhead while preserving the benefit of typo-tolerance where it matters most.

We've seen platforms where fuzzy matching was enabled by default across 20+ fields. Queries that should return in 50ms took 2-3 seconds. Disabling fuzzy on low-value fields dropped latency by 80% with no loss of relevance.

Never measuring results

This is the meta-mistake. You can have perfect boosts, synonyms, and fuzzy logic—but if you're not measuring relevance, you're flying blind.

Relevance measurement takes forms. The simplest is instrumentation: log search queries and which results users click or purchase from. A click or conversion signals relevance. No clicks on the top 5 results signals a bad ranking. Track this at scale:

• Click-through rate (CTR): What percentage of search sessions include a click? Low CTR means results aren't compelling.
• Conversion rate: What percentage of search sessions lead to a purchase or action? This is your ultimate metric.
• Abandonment: How often do users search, see nothing relevant, and leave? High abandonment is a red flag.

Set up basic event tracking in your application:

{
  "event_type": "search",
  "query": "blue shoes",
  "result_count": 147,
  "user_clicked": true,
  "clicked_position": 3,
  "clicked_item_id": "prod_12345",
  "timestamp": "2024-01-15T10:30:00Z"
}

Over weeks, you'll see patterns. "blue shoes" searches have a 65% CTR with current rankings. "winter coat" searches have a 22% CTR. That signals "blue shoes" rankings are working, but "winter coat" needs tuning. You now have data to drive decisions instead of guesses.

The hard part is acting on data systematically. Create a process: measure → identify problem areas → adjust boosts/synonyms → remeasure. A/B test ranking changes when possible. One team improved their e-commerce conversion by 8% just by tracking which exact boost values drove clicks, then iterating weekly.

Without measurement, you're changing configuration blindly. With measurement, every adjustment is informed by user behavior.

Common mistakes

We see these patterns repeatedly across production systems:

• Setting field boosts without business input: Engineers guess at boost values instead of asking "what should users find first?"
• Configuring synonyms once and ignoring them: Synonym lists rot. New products, new terminology, new user intent—none of it flows back into synonyms.
• Applying fuzzy matching everywhere: The performance cost is real, and the signal-to-noise ratio drops as you broaden fuzzy application.
• Launching search and moving on: Relevance isn't a one-time configuration. It requires continuous iteration based on metrics.
• Conflating relevance with recall: A query that returns 10,000 results is not more relevant than one returning 100 good ones. Measure what users actually want.

Key takeaways

• Field boosts are non-negotiable. Name matches should rank higher than description matches. Start with conservative values (2-5x) and iterate.
• Build a maintainable synonym system from day one. Hardcoded synonym lists in code become technical debt fast. Use Elasticsearch's analyzer configuration and audit synonyms quarterly.
• Use fuzzy matching sparingly. Apply it to short fields (names, SKUs) at low edit distances. Skip it on long text and exact identifiers.
• Instrument search with click and conversion tracking. Without metrics, you're tuning in the dark. With metrics, every change is data-driven.
• Treat relevance as an ongoing discipline, not a feature to ship. Production search degrades over time if you don't actively maintain and measure it.