Searchable snapshots in Elasticsearch & Search Engineering

Introduction

By day 94 of our search engineering journey, you've likely encountered the cost-versus-performance trade-off that becomes acute as your data volumes scale. Searchable snapshots in Elasticsearch & Search Engineering offer a compelling solution: the ability to query data stored directly in object storage like Amazon S3 or Google Cloud Storage, bypassing the need to keep everything on expensive local disk. This technique lets you maintain queryable archives and cold data tiers at a fraction of the cost, making it invaluable for compliance-heavy industries and long-term data retention scenarios.

Cold and Frozen Tiers

Elasticsearch's data tiering strategy arranges indices across hot, warm, cold, and frozen tiers, each suited to different access patterns and costs. The cold tier stores older data that's rarely accessed but must remain available; the frozen tier takes this further, using searchable snapshots to query data that lives almost entirely in object storage.

When you move an index to the frozen tier using searchable snapshots, Elasticsearch mounts a snapshot as a searchable index without requiring full restoration to local disk. A node can hold a small "working set" in memory and cache, while the bulk of data remains in S3 or equivalent. Queries are slower because each shard must fetch data on-demand, but the cluster doesn't need the disk capacity or the weeks-long restore time that traditional snapshot recovery would require.

Here's a practical migration scenario: imagine you have a seven-year compliance archive of transaction logs, indexed with daily indices. Year one through six live on cold tier nodes with adequate local storage. Year seven sits frozen on S3. When an auditor requests a query spanning all seven years, your cluster:

• Queries the hot/warm/cold indices normally against local disk
• Simultaneously queries the frozen indices mounted from snapshots
• Merges and returns results transparently

The frozen year takes longer per query, but you're not storing terabytes of old data on expensive NVMe anymore.

Much Cheaper Storage

The economics are straightforward. A local NVMe drive in production nodes costs roughly $0.10–0.30 per GB per month when you factor in the node instance price, replication overhead, and operational burden. Object storage like S3 Glacier or Google Cloud Storage Archive costs $0.004–0.01 per GB per month. That's a 20–50x reduction.

For indices you'll query fewer than a few times per month, searchable snapshots transform your TCO. A 500 GB index queried twice monthly no longer demands a dedicated cold node; it lives as a snapshot in object storage and mounts on-demand.

Consider a SaaS platform storing customer analytics indices. A customer with a 100 GB index from six months ago might generate a report once every three weeks. With searchable snapshots:

• Old approach: Cold tier node with 2 TB storage, ~$300/month, idle most of the time
• New approach: S3 Glacier snapshot, ~$2/month storage, $0.50–2 per query in retrieval and compute

The breakeven is immediate for many workloads. You free up physical infrastructure, reduce your node count, and scale your retention windows without proportional cost increases.

Slower Queries

The trade-off is latency. Querying a frozen index that's mounted from a snapshot is inherently slower than querying a warm index on local NVMe. Why? Because Elasticsearch must fetch data from object storage on-demand, decompress it, and serve results. A typical latency profile:

• Hot/warm index: 50–200 ms for a point lookup or aggregation
• Cold index (on local disk): 100–500 ms
• Frozen index (snapshot mount): 500 ms–5+ seconds, depending on data pattern, network latency, and cache efficiency

The performance degradation is most pronounced on the first query after a shard mounts, before any data is cached locally. Subsequent queries on the same shard benefit from caching; Elasticsearch keeps a working set in memory and on disk. A second query on the same frozen index may run 50–80% faster than the first.

For real-time dashboards, frozen indices are unsuitable. For batch reporting, compliance queries, or ad-hoc investigations into historical data, the latency is acceptable and often welcome because the cost savings are so dramatic.

Here's a practical consideration: if your SLA allows queries to take 5–10 seconds, frozen indices are viable. If you need sub-second latency on every query, keep that data on cold tier with local storage, or move it to warm tier if it's queried frequently.

Great for Compliance Archives

Financial services, healthcare, and e-commerce platforms live under regulatory regimes that mandate retention of transaction logs, audit trails, and customer data for years. Searchable snapshots eliminate the false choice between storing terabytes of data on expensive cluster nodes or deleting it to reduce costs.

A fintech platform must retain order and settlement logs for seven years. That's millions of indices if you create daily indices, and the query volume on year-six data is near zero. With searchable snapshots:

• All seven years of data remain queryable for compliance audits
• Only the current year and previous year live on warm/cold tiers
• Older years live as snapshots on object storage, mounted only when explicitly queried
• When a regulator requests records from three years ago, you execute the query (slowly, but successfully) without restoring or re-ingesting the archive

The compliance benefit extends to immutability. Object storage is write-once-read-many (WORM) in most configurations, meaning historical data cannot be altered after upload. This aligns perfectly with audit requirements and reduces the risk of accidental data loss or tampering.

A healthcare provider indexing patient interaction logs can similarly partition: recent interactions on hot/warm tiers for real-time querying, older interactions on cold tier for frequent but non-urgent lookups, and ancient records on frozen tiers via searchable snapshots. A patient request for their complete history can be fulfilled by querying all tiers, even if it takes minutes to retrieve records from the oldest snapshot.

Common Mistakes

Assuming all queries will be fast: Teams often deploy searchable snapshots, then are shocked when reporting queries take 10x longer. Set explicit SLAs and document that frozen queries are best-effort, not time-critical. Monitor query performance and alert when frozen index queries exceed thresholds, so you catch surprise slowdowns early.

Neglecting cache configuration: Searchable snapshots rely on local caching to serve subsequent queries efficiently. If you under-provision disk space for the cache or misconfigure the working-set size, performance plummets even for repeated queries. Size your cache layer explicitly and measure hit rates.

Mixing cold and frozen inappropriately: Some teams freeze data that should stay cold. Cold tier is for data you'll query multiple times per month; frozen is for data queried a few times per year or on-demand only. Misalignment leads to either unexpectedly slow queries or wasted disk space on cold nodes.

Ignoring object storage costs: While searchable snapshots reduce storage costs, they increase egress and API request costs if queries are frequent or retrieve large result sets. Monitor your S3 or GCS bills for surprise spikes when query patterns change.

Not testing query latency under load: Snapshot queries can compete for network bandwidth and I/O with other cluster operations. Load test your frozen indices under realistic concurrent query load before deploying to production.

Key Takeaways

• Searchable snapshots allow you to query data stored in object storage without restoring it to local disk, eliminating the restore-time bottleneck and enabling true cold-to-frozen tiering.
• Cost savings are dramatic: object storage costs 20–50x less per GB than local NVMe, making long-term retention economically feasible for large indices.
• Query latency on frozen indices is significantly higher (500 ms–5+ seconds) compared to hot/warm indices, making them unsuitable for real-time dashboards but ideal for batch reporting and compliance queries.
• Frozen indices excel in compliance and regulatory scenarios where data must be retained and queryable but is accessed infrequently; audit trails, transaction logs, and historical records are prime candidates.
• Configure caching thoughtfully, monitor object storage egress costs, and test query performance under load to avoid surprises in production.
• Use cold tier for data queried multiple times monthly, and frozen tier for data queried a few times yearly or on-demand; mixing the two wastes resources or compromises latency.