Choosing a search architecture: best practices explained

Choosing a search architecture: best practices explained

The moment you add a search box to your product, you're making an architectural decision that will shape how your system scales for years. Many teams rush through this choice, only to rebuild their search stack mid-flight when the database can no longer keep up. In this article, we walk through the lifecycle of search architecture—from MVP simplicity to enterprise-grade complexity—so you can make informed decisions at each growth stage without overengineering too early or leaving performance on the table.

Understanding the search architecture lifecycle

There's no single "right" search stack. Instead, there are pragmatic choices that fit your current scale, team capacity, and revenue situation. The key is recognizing when to evolve. We've seen teams waste months optimizing PostgreSQL full-text search when they should have migrated to Elasticsearch, and others spin up complex distributed search clusters when a basic index would suffice.

The architecture you choose today should solve your problem today—not the problem you hope to have in two years. That said, you should architect toward the next stage, meaning your data model and indexing patterns should port cleanly when you upgrade.

Stage 1: MVP — Database search or a hosted engine

When you launch, your dataset is small. Your product might have 10,000 products, 50,000 documents, or a few million rows. At this scale, built-in database search is often the right choice because it requires almost no infrastructure work.

Database-native search

PostgreSQL's full-text search engine is surprisingly capable. You index text columns, tune your queries, and you're done. No extra services to deploy, monitor, or pay for. This works well if:

• Your dataset is under 5–10 million records.
• You tolerate 200–500ms query latency (acceptable for many use cases).
• Your search patterns are simple: keyword matching and basic filtering.
• You have one primary language, or you can accept basic tokenization.

Here's a minimal PostgreSQL full-text search setup:

-- Add a tsvector column
ALTER TABLE products ADD COLUMN search_vector tsvector;

-- Index it
CREATE INDEX products_search_idx ON products USING gin(search_vector);

-- Create a trigger to keep it up-to-date
CREATE TRIGGER products_search_update BEFORE INSERT OR UPDATE ON products
FOR EACH ROW EXECUTE FUNCTION tsvector_update_trigger(search_vector, 'pg_catalog.english', name, description);

-- Query it
SELECT * FROM products
WHERE search_vector @@ plainto_tsquery('english', 'leather jacket')
ORDER BY ts_rank(search_vector, query) DESC;

The trade-off is clear: your search is bound to your transactional database. Any complex faceting, real-time scoring, or internationalization will require application logic.

Managed search engines

If you want to decouple search immediately, consider a hosted Elasticsearch service like Elastic Cloud, AWS OpenSearch, or Algolia. These handle infrastructure and scaling for you. Algolia, in particular, is purpose-built for product search and gives you excellent defaults for relevance and faceting with minimal configuration.

The cost is higher (typically $20–50 per month at small scale), but you gain:

• Tunable relevance: field boosts, exact-match scoring, synonym handling.
• Rich filtering: facets, range queries, nested filters.
• Better latency: typically 50–100ms for well-tuned queries.
• Horizontal scalability baked in.

This stage works for roughly 12–18 months of typical growth. Use it to validate that search is core to your product and to build muscle around relevance tuning.

Stage 2: Growth — Dedicated Elasticsearch with sync pipeline

Your dataset has grown to 50 million records, or your search is now integral to revenue. Your managed service bill is climbing, or you need features that Algolia doesn't provide (like exact-phrase scoring or custom analyzers). It's time to run Elasticsearch in production.

Setting up the sync pipeline

The critical decision here is how you keep Elasticsearch in sync with your database. We typically see two patterns: event-driven and batch-driven.

Event-driven (recommended): Whenever you update a product in PostgreSQL, you emit an event to a message queue (Kafka, SNS/SQS, or even a simple webhook). A consumer service picks up that event and updates Elasticsearch immediately. This gives you near-real-time search and is naturally resilient to network blips.

// After updating a product in PostgreSQL
await db.products.update(productId, { name: 'New Name' });

// Emit event
await messageQueue.publish('product.updated', {
  id: productId,
  name: 'New Name',
  timestamp: Date.now()
});

// Consumer service listens and updates Elasticsearch
messageQueue.subscribe('product.updated', async (msg) => {
  await esClient.index({
    index: 'products',
    id: msg.id,
    body: {
      name: msg.name,
      updatedAt: msg.timestamp
    }
  });
});

The event approach works best because it decouples your application from search. If Elasticsearch is temporarily down, events queue up and are processed when it comes back online.

Batch-driven: Run periodic jobs (every 5 minutes, hourly, or nightly) to sync changed records from PostgreSQL into Elasticsearch. This is simpler to implement but introduces latency—your search results lag behind your database by that interval. It's acceptable if your search doesn't power critical business flows (e.g., internal admin dashboards, not user-facing discovery).

Indexing strategy

At this stage, you typically have one index per entity type (products, articles, users). Keep your mapping lean: include only fields you actually search or filter on. Store heavy payloads (HTML content, image metadata) in your database, not Elasticsearch.

{
  "mappings": {
    "properties": {
      "id": { "type": "keyword" },
      "name": {
        "type": "text",
        "analyzer": "standard",
        "fields": {
          "keyword": { "type": "keyword" }
        }
      },
      "category": { "type": "keyword" },
      "price": { "type": "float" },
      "created_at": { "type": "date" }
    }
  }
}

This stage is sustainable for 2–4 years if your data grows steadily. The index sits on a single or small cluster, you understand your query patterns, and you can afford modest operational overhead to monitor and reindex as needed.

Stage 3: Scale — Separate indexes, aliases, and ILM

Your index has grown to hundreds of gigabytes. Query latency is creeping up. You're running multiple Elasticsearch nodes and thinking about sharding strategy. At this point, you need discipline around index lifecycle management.

Index rotation with aliases

Stop thinking of "the products index" as a single, permanent artifact. Instead, treat it as a logical entity backed by versioned indexes: products-v1, products-v2, and so on. Use Elasticsearch aliases to point users to the current index.

This pattern lets you reindex without downtime. When you need to change your mapping or add a new field, you create a new index with the updated mapping, sync data into it, test it thoroughly, then atomically swap the alias to point to the new index. The old index stays around for a rollback window, then you delete it.

// Create a new index with updated mapping
PUT /products-v2
{
  "mappings": {
    "properties": {
      "id": { "type": "keyword" },
      "name": { "type": "text" },
      "category": { "type": "keyword" },
      "price": { "type": "float" },
      "inventory": { "type": "integer" }  // New field
    }
  }
}

// Reindex from old to new
POST /_reindex
{
  "source": { "index": "products-v1" },
  "dest": { "index": "products-v2" }
}

// Test queries against v2...

// Swap alias atomically
POST /_aliases
{
  "actions": [
    { "remove": { "index": "products-v1", "alias": "products" } },
    { "add": { "index": "products-v2", "alias": "products" } }
  ]
}

// Keep v1 for 24 hours, then delete
DELETE /products-v1

Index lifecycle management (ILM)

If your data is time-series (logs, events, metrics), use Elasticsearch's built-in ILM policies to automate index rollover and retention. Define policies that move indexes through phases: hot (recent data, optimized for ingestion), warm (less frequently accessed), cold (archived), and delete (purged after a retention period).

PUT _ilm/policy/events_policy
{
  "policy": "events_policy",
  "phases": {
    "hot": {
      "min_age": "0ms",
      "actions": {
        "rollover": {
          "max_primary_store_size": "50gb",
          "max_age": "1d"
        }
      }
    },
    "warm": {
      "min_age": "7d",
      "actions": {
        "set_priority": { "priority": 50 }
      }
    },
    "delete": {
      "min_age": "30d",
      "actions": {
        "delete": {}
      }
    }
  }
}

At scale, you're also thinking about cluster topology. You likely have dedicated master nodes (managing cluster state), data nodes (storing shards), and ingest nodes (preprocessing documents). This complexity is justified once your index is multiple terabytes or your query volume exceeds thousands per second.

Stage 4: AI search — Hybrid and vector retrieval

As your product matures, customers expect smarter search. They want to find products by describing them ("waterproof hiking boots") rather than exact keywords. They want semantic similarity ("show me shoes like this one"). This is where AI-powered search enters the picture.

Vector embeddings

Modern search stacks combine keyword matching with semantic search using dense vector embeddings. You generate embeddings for your documents using a model (OpenAI's text-embedding-3-small, Cohere, or open-source alternatives), store them in Elasticsearch, and query using vector similarity.

// Add a vector field to your mapping
PUT /products-v3
{
  "mappings": {
    "properties": {
      "id": { "type": "keyword" },
      "name": { "type": "text" },
      "description": { "type": "text" },
      "embedding": {
        "type": "dense_vector",
        "dims": 1536,
        "index": true,
        "similarity": "cosine"
      }
    }
  }
}

// Index a document with embedding
POST /products-v3/_doc
{
  "id": "shoe-001",
  "name": "Waterproof Hiking Boot",
  "description": "Rugged, waterproof leather boot with ankle support",
  "embedding": [0.123, -0.045, 0.067, ...]  // 1536-dim vector from embedding API
}

Hybrid queries

You query both the keyword index and the vector index simultaneously, then blend the results. A hybrid approach typically performs better than either alone because keyword search catches exact and near-exact matches, while vector search finds semantic neighbors.

POST /products/_search
{
  "query": {
    "bool": {
      "should": [
        {
          "multi_match": {
            "query": "waterproof hiking boots",
            "fields": ["name^2", "description"]
          }
        },
        {
          "knn": {
            "field": "embedding",
            "query_vector": [0.12, -0.04, 0.07, ...],  // Embedding of user query
            "k": 10,
            "num_candidates": 100
          }
        }
      ]
    }
  }
}

The challenge at this stage is operational: generating and updating embeddings for millions of documents, managing embedding freshness, and tuning the balance between keyword and vector scores. Most teams use a dedicated service to batch-generate embeddings offline, then sync them into Elasticsearch alongside your product data.

Re-ranking and personalization

With enough data and complexity, you may add a learning-to-rank layer. Collect signals on which results users click or convert on, train a model to predict relevance, and use that model to re-rank your Elasticsearch results before returning them to the user. This requires significant infrastructure (data pipelines, model training, feature stores), so it's only worth it if search is genuinely a competitive advantage for your business.

Common mistakes

• Over-engineering the MVP. Don't spin up a multi-node Elasticsearch cluster with ILM and vector embeddings on day one. Use PostgreSQL or a managed service, learn the domain, then upgrade.
• Neglecting sync consistency. Event-driven sync looks elegant but is fragile if you don't handle failures, retries, and idempotency. Start simple (periodic batch syncs) and evolve if latency demands it.
• Ignoring relevance tuning. Most teams optimize infrastructure before tuning scoring. Spend weeks on field boosts, synonym lists, and A/B testing relevance before scaling the cluster.
• Forgetting monitoring. Without alerting on query latency, indexing lag, and shard health, you'll discover problems after your users do.
• Premature vectorization. Vector search is powerful but expensive to compute and maintain. Validate that semantic search actually improves your metrics before building it into the core loop.

Key takeaways

• Start with database-native search or a managed service. You need real usage data to justify complexity.
• When you self-host Elasticsearch, prioritize a robust sync pipeline—event-driven is preferable but requires careful error handling.
• Use aliases and index rotation to enable zero-downtime reindexing and schema evolution.
• Hybrid search (keyword + vector) outperforms either approach alone and is worth the operational cost once search is core to your business.
• Measure and monitor at every stage. Relevance tuning and operational health matter more than feature count.