Securing Elasticsearch: a practical guide for developers

Introduction

If you've got an Elasticsearch cluster running in production, you've got a target on your back. We've seen too many teams discover their data exposed to the internet—or worse, ransomed—because security felt like a nice-to-have rather than table stakes. Securing Elasticsearch: a practical guide for developers isn't about achieving perfect compliance; it's about shipping production search safely, starting today.

The good news: Elasticsearch 8.x made security the default. TLS, authentication, and role-based access control come out of the box. The challenge is that many of us still run older clusters, upgrade without enabling these features, or misconfigure them in ways that leave gaps. This guide walks you through the practical steps we've taken on real projects—e-commerce platforms, SaaS backends, trading systems—to lock down clusters without breaking developer workflows.

Enable Security: TLS and Authentication Are On By Default in 8.x

Starting with Elasticsearch 8.0, security is enabled by default. If you're spinning up a new cluster, you get X-Pack security, TLS, and an auto-generated password for the elastic superuser. But "default" doesn't mean "done." You need to understand what's actually happening and verify it's working.

TLS configuration

Every connection between your application and Elasticsearch should be encrypted. In 8.x, this is handled by X-Pack Security, and certificates are generated during cluster initialization. When you first start a new node, Elasticsearch creates a CA, generates node certificates, and stores them in config/certs/.

For a development cluster on your machine, this just works. For production, you need to think about certificate rotation and distribution. Here's what we typically do:

{
  "xpack.security.enabled": true,
  "xpack.security.enrollment.enabled": true,
  "xpack.security.http.ssl.enabled": true,
  "xpack.security.http.ssl.keystore.path": "certs/http.p12",
  "xpack.security.http.ssl.keystore.password": "${KEYSTORE_PASSWORD}",
  "xpack.security.transport.ssl.enabled": true,
  "xpack.security.transport.ssl.keystore.path": "certs/transport.p12",
  "xpack.security.transport.ssl.keystore.password": "${KEYSTORE_PASSWORD}"
}

The two SSL configurations matter:

• HTTP SSL: Encrypts traffic between your app and the REST API (port 9200).
• Transport SSL: Encrypts node-to-node communication (port 9300).

Both should be enabled. Transport SSL is especially easy to overlook because it doesn't affect external clients, but a compromised node can otherwise read unencrypted inter-node traffic.

Authentication and the elastic user

When you first initialize a cluster, Elasticsearch prints an enrollment token and a password for the elastic user. Store that password in your secrets manager immediately—it's your superuser account. Don't hardcode it anywhere.

For local development or testing, you can reset it:

bin/elasticsearch-reset-password -u elastic -i

The -i flag makes it interactive. In Docker or Kubernetes, you'd often set the password via an environment variable during initialization.

Here's a Docker Compose example we've used on teams:

services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.10.0
    environment:
      - discovery.type=single-node
      - ELASTIC_PASSWORD=YourSecurePasswordHere123!
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.verification_mode=certificate
    ports:
      - "9200:9200"
    volumes:
      - ./certs:/usr/share/elasticsearch/config/certs

Once authentication is on, every request needs credentials. Your application can't connect without them.

Roles: Least-Privilege Roles Per App

Here's where security becomes an ongoing practice. You could log everything in as elastic, but you shouldn't. Each application or service should have its own role with only the permissions it needs.

The principle of least privilege

Imagine you have a search microservice, an analytics pipeline, and an admin dashboard. Each has different needs:

• The search service reads from products-* indices and writes to a cache index.
• The analytics pipeline reads all indices and writes aggregation results.
• The admin dashboard manages index lifecycle policies and can delete old logs.

If all three use the elastic user, a breach of any one gives an attacker full cluster access. Instead, create a role for each:

{
  "roles": [
    {
      "role": "search_app",
      "indices": [
        {
          "names": ["products-*"],
          "privileges": ["read", "view_index_metadata"]
        },
        {
          "names": ["cache-search"],
          "privileges": ["read", "write", "index"]
        }
      ],
      "applications": [],
      "run_as": [],
      "metadata": {},
      "translog": {
        "durability": "request"
      }
    }
  ]
}

This role can read from any index matching products-*, and read/write to cache-search. It can't delete indices, manage settings, or access other data. If that application's credentials are compromised, the blast radius is small.

Creating roles via the API

You can create roles through the security API. We typically do this during deployment or as part of an infrastructure-as-code setup:

PUT _security/role/search_app
{
  "indices": [
    {
      "names": ["products-*"],
      "privileges": ["read", "view_index_metadata"]
    },
    {
      "names": ["cache-search"],
      "privileges": ["read", "write", "index"]
    }
  ]
}

Then create a user for that role:

POST _security/user/search_app_user
{
  "password": "ChangeThisToAStrongPassword123!",
  "roles": ["search_app"],
  "full_name": "Search Application User",
  "email": "ops@company.dev"
}

From here, the search microservice uses search_app_user credentials instead of elastic.

Common role patterns

We've found a few roles cover most scenarios:

• read_only: Read from specific indices, view metadata. For dashboards or reporting tools.
• ingest: Write to indices, manage their own templates. For data pipelines.
• admin: Full cluster management. Rarely needed; keep to a few ops team members.
• monitor: Read cluster health and metrics without access to data. For monitoring tools.

Define these once, grant them to users or API keys, and reuse them across environments.

API Keys: Scoped Keys for Services

API keys are our preferred way to authenticate services. They're simpler than username/password, easier to rotate, and you can scope them narrowly—one key per service, one key per environment.

Why API keys over username/password

API keys have several advantages:

• They can't be used to log into the web UI (reducing surface area).
• You can set an expiration date and revoke them instantly.
• You can create multiple keys per application without managing multiple password policies.
• Logs show which key was used, aiding audit trails.

Creating and using API keys

Create a key via the API:

POST _security/api_key
{
  "name": "search-indexer-prod",
  "role_descriptors": {
    "indexer_role": {
      "indices": [
        {
          "names": ["products-*"],
          "privileges": ["write", "index", "create_index"]
        }
      ]
    }
  },
  "expiration": "90d"
}

Elasticsearch returns an ID and encoded key:

{
  "id": "abc123def456",
  "api_key": "encoded_key_here_base64",
  "encoded": "YWJjMTIzZGVmNDU2OmVuY29kZWRfa2V5X2hlcmVfYmFzZTY0"
}

Store the encoded value securely. To use it, pass it in the Authorization header:

curl -H "Authorization: ApiKey encoded_key_here_base64" \
  https://your-elasticsearch:9200/products-2024-01/_doc

In code, it looks like this (Node.js example):

const client = new Client({
  node: 'https://your-elasticsearch:9200',
  auth: {
    apiKey: 'abc123def456:encoded_key_here_base64'
  }
});

Rotating API keys

API keys can expire. Set an expiration when you create them—90 days is common for production services. When an expiration date approaches, generate a new key, update your application config, and revoke the old one:

DELETE _security/api_key
{
  "ids": ["abc123def456"]
}

Automate this. Set calendar reminders or use your deployment pipeline to flag keys expiring within 30 days.

Network: Private Subnets, Never Public Endpoints

Even with TLS and authentication enabled, exposing Elasticsearch to the internet is a mistake. We've seen clusters found via Shodan and attacked within hours.

Elasticsearch in private subnets

Run your cluster in a private subnet—no inbound routes from the internet. In AWS, that means a subnet without a route to an internet gateway. In Kubernetes, that means no external load balancer on port 9200.

Applications in the same VPC or cluster can reach it via private DNS. External tools (backups, monitoring) access it through a bastion host or VPN.

Firewall rules

Even within private subnets, restrict network access further. Only services that need Elasticsearch should be able to reach it:

• Search microservice: allowed.
• Analytics service: allowed.
• Random developer laptop: not allowed (use a bastion or VPN).
• Production database: not allowed.

In Kubernetes, use a NetworkPolicy. In AWS, use security groups:

{
  "GroupId": "sg-elasticsearch",
  "IpPermissions": [
    {
      "IpProtocol": "tcp",
      "FromPort": 9200,
      "ToPort": 9200,
      "UserIdGroupPairs": [
        {
          "GroupId": "sg-search-service",
          "Description": "Search microservice access"
        },
        {
          "GroupId": "sg-analytics-service",
          "Description": "Analytics pipeline access"
        }
      ]
    }
  ]
}

Bastion hosts and VPN

Developers and ops need to connect for debugging, maintenance, and cluster administration. Never give them direct internet access. Use a bastion host (a hardened jump server) or VPN:

# Connect through bastion
ssh -L 9200:elasticsearch.internal:9200 bastion@bastion.prod.company.dev

# Then locally
curl -u elastic:password https://localhost:9200/_cluster/health

This keeps Elasticsearch private while allowing authorized access for legitimate use.

Common Mistakes

• Upgrading without enabling security: If you're running Elasticsearch 7.x or older, security isn't on by default. Don't assume it's enabled; check xpack.security.enabled in your config.
• Sharing the elastic user password: Every team member logging in as elastic means audit trails are useless, and you can't revoke access without changing the password for everyone.
• Creating API keys without expiration: A leaked key that never expires is leaked forever. Always set an expiration and rotate periodically.
• Overly broad roles: A role that can read all indices and write to all indices defeats the purpose of security. Be specific: which indices? which privileges?
• Forgetting about inter-node communication: Turning on HTTP SSL but leaving transport SSL off means cluster traffic is still unencrypted and readable to anyone on your network.
• Exposing the cluster on a public load balancer: Even with strong auth and TLS, a public endpoint is an attack magnet. Keep it private.

Key Takeaways

• Enable TLS and authentication immediately. In 8.x, it's default; in older versions, turn on xpack.security.enabled: true.
• Create a role per application with least-privilege access. Never share the elastic superuser credentials outside your ops team.
• Use API keys for service authentication. Set expiration dates and rotate them regularly—90 days is a reasonable baseline.
• Run Elasticsearch in a private subnet. No public endpoints. Use bastion hosts or VPNs for administrative access.
• Audit your clusters regularly. Check which users exist, which roles they have, and when API keys expire. Make this part of your weekly ops checklist.
• Security is a practice, not a checkbox. As your application grows, review and tighten permissions. A breach often happens not because the tools are weak, but because they were never fully deployed or maintained.