Building site search for a blog: lessons from production
When your blog grows from dozens to thousands of articles, basic SQL queries stop working. We learned this the hard way after watching search latency climb from milliseconds to seconds and relevance scores become meaningless. Building site search for a blog sounds simple until you actually ship it to production, encounter real query patterns, and realize that relevance tuning is as much art as engineering.
In this article, we'll walk through a production-grade site search implementation using Elasticsearch. We've built similar systems across travel platforms, SaaS documentation portals, and e-commerce content hubs, and the patterns here carry forward. This is about pragmatic choices—what to index, how to rank, what to highlight—that make the difference between a search feature users tolerate and one they actually rely on.
Why Elasticsearch for blog search?
Before we dive into the implementation, let's be clear: you don't always need Elasticsearch. If your blog has under 10,000 articles and search traffic is minimal, a PostgreSQL full-text search might suffice. But once you need relevance tuning, faceted navigation, typo tolerance, or sub-second response times at scale, a dedicated search engine pays for itself.
Elasticsearch gives us inverted indexes, relevance scoring out of the box, and the flexibility to tune ranking without redeploying your application. For a blog, that means you can A/B test boosting strategies, adjust decay functions based on user behavior, and iterate without touching your backend business logic.
Index articles—Title, summary, body, tags and date
Your index structure determines what you can search and how efficiently. We'll design a document that captures the core metadata about each blog post while staying performant.
Here's our mapping:
{
"mappings": {
"properties": {
"id": {
"type": "keyword"
},
"title": {
"type": "text",
"analyzer": "standard",
"fields": {
"raw": {
"type": "keyword"
}
}
},
"summary": {
"type": "text",
"analyzer": "standard"
},
"body": {
"type": "text",
"analyzer": "standard"
},
"tags": {
"type": "keyword"
},
"published_at": {
"type": "date"
},
"view_count": {
"type": "integer"
},
"author": {
"type": "keyword"
}
}
}
}Notice the structure. Title, summary, and body use the text analyzer because we want them tokenized and searched. Tags are keywords—we filter by them exactly, no tokenization. Published_at is a date field, which lets us apply recency decay functions later. View_count is numeric; we'll use it for optional popularity boosting.
When indexing a new article, you'd send a document like this:
{
"id": "article-123",
"title": "Building scalable Kafka consumers with NestJS",
"summary": "A guide to handling backpressure and error recovery in high-throughput message processing.",
"body": "Kafka consumers often fail silently... [full article text]",
"tags": ["kafka", "nestjs", "production"],
"published_at": "2024-01-15T10:00:00Z",
"view_count": 4200,
"author": "team"
}Store the full body. Yes, it inflates your index size, but it enables highlighting (which we'll cover next) and provides context for relevance scoring. If storage is tight, you can use compression or store a truncated version, but we've found the trade-off rarely pays off.
Boost titles—Title matches rank highest
Not all matches are equal. An exact title match should rank above a body match. Users searching for "Kafka consumers" expect articles with that phrase in the title to appear first.
We use a multi-field query with boosting:
{
"query": {
"multi_match": {
"query": "Kafka consumers",
"fields": [
"title^3",
"summary^2",
"body",
"tags^2"
],
"type": "best_fields",
"operator": "or"
}
}
}The ^3 syntax means title matches get 3x the relevance score of body matches. Summary gets a 2x boost because it's a curated snippet. Tags also get a 2x boost—matching a tag is usually intentional filtering. The body has no boost (coefficient of 1), so it contributes but doesn't dominate.
The best_fields type means Elasticsearch finds the single best-matching field and uses that score, rather than summing all field scores. This prevents documents with the query scattered across multiple fields from artificially inflating.
In practice, adjust these numbers based on user feedback. If users complain that body matches rank too high, increase the title boost. If tag matches feel irrelevant, lower their boost. Start conservative—a 2x to 4x range works for most blogs—and iterate.
Highlight—Show matching snippets
Showing users why a result matched builds trust. Without highlighting, a search result is just a title and summary; with highlighting, users see the exact phrase they searched for in context.
Add a highlight clause to your query:
{
"query": {
"multi_match": {
"query": "Kafka consumers",
"fields": ["title^3", "summary^2", "body", "tags^2"],
"type": "best_fields",
"operator": "or"
}
},
"highlight": {
"pre_tags": ["<em>"],
"post_tags": ["</em>"],
"fields": {
"title": {},
"summary": {},
"body": {
"fragment_size": 150,
"number_of_fragments": 3
}
}
}
}Elasticsearch wraps matching terms in your specified tags. For the body, we limit to 3 fragments of 150 characters each—this prevents returning the entire 5,000-word article and keeps response payload manageable. For title and summary, we return the whole field since they're smaller.
When rendering results, display the highlight HTML if present; otherwise fall back to the original field. Here's a pseudocode example:
{
"hits": [
{
"_id": "article-123",
"_score": 2.847,
"_source": {
"title": "Building scalable Kafka consumers with NestJS",
"summary": "A guide to handling backpressure..."
},
"highlight": {
"body": [
"...robust error handling. <em>Kafka consumers</em> often fail silently...",
"...Dead Letter Queues. <em>Consumers</em> should persist offset state..."
]
}
}
]
}Users see the highlighted body fragments alongside the title. This context dramatically reduces bounce rates on search results—people don't click blind anymore.
Recency—Decay functions favor recent posts
Older articles are often still relevant, but newer posts typically rank higher. Without a recency factor, a five-year-old article matching your query might score the same as yesterday's. Decay functions solve this by gradually reducing the score of older documents without excluding them entirely.
Use a function_score query with Elasticsearch's gauss decay function:
{
"query": {
"function_score": {
"query": {
"multi_match": {
"query": "Kafka consumers",
"fields": ["title^3", "summary^2", "body", "tags^2"],
"type": "best_fields"
}
},
"functions": [
{
"gauss": {
"published_at": {
"origin": "now",
"scale": "30d",
"decay": 0.5
}
}
}
],
"boost_mode": "multiply"
}
}
}This setup says: "An article published today scores 100%, one published 30 days ago scores 50% (the decay value), and older articles score progressively lower." The multiply boost mode means the decay function multiplies the base relevance score, so a high-relevance old article still ranks well, but a low-relevance recent article won't get an artificial boost.
Adjust the scale (30d) based on your content velocity. A high-frequency publication like a daily technical newsletter might use 7d. A quarterly research blog might use 90d. The decay parameter (0.5) controls how aggressively recency affects ranking. Lower values (0.1 to 0.3) heavily favor recent posts; higher values (0.6 to 0.9) are more forgiving to older content.
In production, we've also seen teams add a view_count function alongside recency, treating popular articles as slightly more relevant:
"functions": [
{
"gauss": {
"published_at": {
"origin": "now",
"scale": "30d",
"decay": 0.5
}
}
},
{
"field_value_factor": {
"field": "view_count",
"modifier": "log1p",
"factor": 0.1
}
}
],
"boost_mode": "multiply"The log1p modifier prevents very high view counts from drowning out relevance. A 10,000-view article gets a modest boost over a 1,000-view article, but relevance still matters most.
Common mistakes
Indexing HTML instead of plain text. If you copy-paste HTML from your CMS into the body field, users will see HTML tags in search results and highlighting will be confusing. Strip or render HTML before indexing.
Over-boosting titles. A title boost above 5x often ranks irrelevant exact-match titles above highly relevant body content. Stay in the 2x to 4x range and test with real queries.
Ignoring the body in highlighting. Returning only title and summary highlights gives users no context. Always highlight the body field even if it's large—Elasticsearch's fragment limits keep it manageable.
Setting decay scale too aggressively. If your scale is 7d but your content is quarterly, nothing older than two months will appear in results. Match scale to your publishing cadence.
Forgetting to handle empty queries. An empty search string will match everything. Decide whether you return all articles (sorted by recency) or return nothing. Most blogs return all, sorted by date and popularity.
Key takeaways
- Store the full article body in your index even if it adds size—it enables highlighting and improves relevance.
- Use multi_match queries with field-specific boosts: title gets the highest, summary and tags get medium, body gets baseline.
- Highlight body fragments with a limit (150 chars, 3 fragments) to keep responses fast and readable.
- Apply recency decay functions with a scale that matches your content velocity; older articles remain searchable but rank lower.
- Test boosting parameters with real user queries and adjust based on feedback—relevance tuning is iterative.
- Monitor query latency and index size; if either grows unsustainably, consider archiving old articles or refining your analyzer settings.
Comments
Loading comments…