Source filtering and stored fields: a step-by-step guide

Introduction

By day 88 of building search systems, you've likely tuned query performance and watched response times drop. But there's still a common bottleneck teams overlook: returning fields you don't actually need. Source filtering and stored fields: a step-by-step guide covers the practical techniques to trim response payloads and keep your Elasticsearch cluster efficient without sacrificing the data your application requires.

Whether you're paginating through millions of e-commerce listings or streaming real-time trading data, every byte matters. In this post, we'll walk through _source filtering, understand when to disable it entirely, and explore docvalue_fields as a lean alternative for sorting and aggregations.

Understanding _source includes and excludes

Every document in Elasticsearch stores an _source field—the original JSON you indexed. By default, Elasticsearch returns the entire _source with every hit. For a product document containing SKUs, descriptions, images, ratings, and internal metadata, that's often kilobytes per record.

The _source parameter lets you be surgical about what comes back. You have two modes: includes (whitelist what you want) and excludes (blacklist what you don't). In production, we've found includes to be safer—you control exactly what leaks into responses.

Here's a real scenario: your ride-hailing platform indexes trips with location history, driver details, and passenger feedback. When listing recent trips in the mobile app, you only need the trip ID, date, fare, and route summary—not the full 2 KB payload.

{
  "query": {
    "match_all": {}
  },
  "_source": {
    "includes": ["trip_id", "date", "fare", "route_summary"]
  }
}

Notice we're using an object with includes array, not a simple string. This syntax is more explicit and easier to maintain across your codebase than the shorthand "_source": ["field1", "field2"].

You can also combine includes and excludes. Suppose you want everything except internal metadata:

{
  "_source": {
    "includes": ["*"],
    "excludes": ["internal_*", "debug_*"]
  }
}

Wildcards work at any level. If your document has nested objects like driver.phone and driver.bank_account, you can target driver.* or be more specific: driver.phone.

Smaller responses and network efficiency

Reducing _source doesn't just save disk I/O—it cuts network bandwidth, which is often the real bottleneck in distributed systems. We've seen teams running Elasticsearch in one AWS region and API servers in another discover that cutting response size from 5 KB to 1.2 KB per document saved 30% on data transfer costs.

The math is simple: if you're returning 100 search results and each shrinks by 4 KB, that's 400 KB saved per request. Multiply by millions of requests per day, and you're looking at meaningful infrastructure savings.

In a travel booking system, when you search for hotel availability, you get back hundreds of results. The search response includes booking URLs, descriptions, amenities, and reviews. But the initial results page only needs titles, prices, ratings, and images—everything else is lazy-loaded on detail view.

{
  "query": {
    "multi_match": {
      "query": "paris hotels",
      "fields": ["name", "city"]
    }
  },
  "_source": ["name", "price", "rating", "image_url"],
  "size": 20
}

Combined with pagination and proper indexing, this keeps responses fast enough for snappy UX. Most modern clients also gzip responses, so removing verbose fields has a compounding effect.

Disable _source only with care

You can set "_source": false during indexing or querying to skip storing or returning _source altogether. This sounds attractive—why store redundant data?—but it's dangerous.

Without _source, you lose the ability to reindex without external systems. When you update an analyzer or add a new field mapping, reindexing requires fetching the original document. If _source is disabled, you're stuck maintaining separate backups or re-querying your primary database.

We learned this the hard way in a SaaS environment where we disabled _source to save space, then needed to reindex during a schema migration. The cost of maintaining external backups far exceeded the storage savings.

There are narrow cases where it makes sense: logging systems that only ever query by IDs or aggregations and never need the full document. Even then, store enough metadata to reconstruct what you need.

Our recommendation: keep _source enabled. Storage is cheap. The flexibility isn't.

Use docvalue_fields for sorting values

When you need a field for sorting or aggregations but not for display, docvalue_fields is your tool. This returns values from Elasticsearch's columnar doc_values storage, which is optimized for sorting and filtering, bypassing _source entirely.

Doc values are stored separately and are already in memory for fast aggregations. For numeric fields, dates, and keywords, they're faster than parsing _source.

Picture an e-commerce platform sorting products by relevance, then by price. The query needs the product name and image for display, but uses price internally for sorting:

{
  "query": {
    "multi_match": {
      "query": "laptop",
      "fields": ["name^2", "description"]
    }
  },
  "_source": ["name", "image_url"],
  "docvalue_fields": ["price", "stock_count"],
  "sort": ["_score", {"price": {"order": "asc"}}],
  "size": 50
}

The response includes fields with values from docvalue_fields, separate from _source. This is particularly useful when:

• Sorting on high-cardinality fields where fielddata would be expensive
• You need exact numeric values for calculations but not for rendering
• Working with arrays or nested fields where you need specific sub-values

One nuance: not all field types support doc values. Text fields don't, but keywords do. Numeric types, dates, and booleans all support them by default. If you enable doc values on text via "fielddata": true, you're defeating the purpose.

Common mistakes

Including too much in _source. Teams often include fields for future flexibility that never get used. Start narrow—add fields only when required. It's easier to expand later than to debug why responses are slow.

Disabling _source permanently. We've seen this done to save 5% storage and lose 50% operational flexibility. Don't do it unless you have tested alternative recovery procedures.

Confusing _source filtering with query filtering. Filtering the query results with bool queries reduces documents processed. Filtering _source doesn't—it still queries everything, just returns fewer fields. Use _source for what to display, not to reduce search scope.

Overusing docvalue_fields for text. If you need a text field, it won't have doc values unless you've explicitly enabled fielddata (which you shouldn't for large text). Use _source or store it as a keyword.

Setting _source: false in mappings, then trying to reindex. This is the cardinal sin. Make it explicit in queries if you're testing, but keep it indexed.

Key takeaways

• Use _source includes to whitelist only fields your application needs, reducing payload size and network costs.
• Combine includes and excludes with wildcards for flexible, maintainable source filtering.
• Smaller responses compound benefits: faster networks, lower costs, better UX.
• Keep _source enabled at the mapping level—disabling it saves negligible storage but breaks reindexing workflows.
• Leverage docvalue_fields for sorting and aggregations on numeric, keyword, and date fields to bypass _source parsing.
• Source filtering and stored fields: a step-by-step guide is fundamentally about intention—request only what you use.