How to Self-Host Elasticsearch with Docker

What Is Elasticsearch?

Elasticsearch is the most widely deployed search and analytics engine in the world. Built on Apache Lucene, it handles full-text search, structured search, analytics, and log aggregation. It’s the core of the ELK Stack (Elasticsearch, Logstash, Kibana) used for observability across millions of deployments. Self-hosting gives you complete control over your search and analytics data.

Prerequisites

A Linux server (Ubuntu 22.04+ recommended)
Docker and Docker Compose installed (guide)
4 GB+ RAM (2 GB minimum for JVM heap)
10 GB+ free disk space
Set vm.max_map_count=262144 on the host (required)

Set the required kernel parameter:

sudo sysctl -w vm.max_map_count=262144
# Make persistent:
echo "vm.max_map_count=262144" | sudo tee -a /etc/sysctl.conf

Docker Compose Configuration

Create a docker-compose.yml file:

services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:9.3.1
    container_name: elasticsearch
    ports:
      - "9200:9200"
    volumes:
      - es_data:/usr/share/elasticsearch/data
    environment:
      # Single-node mode (no cluster discovery)
      - discovery.type=single-node
      # JVM heap size (set to ~50% of available RAM, max 32g)
      - ES_JAVA_OPTS=-Xms1g -Xmx1g
      # Disable security for development (enable for production)
      - xpack.security.enabled=false
      # Cluster name
      - cluster.name=selfhosted-search
      # Node name
      - node.name=es-node-1
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
    restart: unless-stopped

volumes:
  es_data:

For production with security enabled:

services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:9.3.1
    container_name: elasticsearch
    ports:
      - "9200:9200"
    volumes:
      - es_data:/usr/share/elasticsearch/data
    environment:
      - discovery.type=single-node
      - ES_JAVA_OPTS=-Xms2g -Xmx2g
      - xpack.security.enabled=true
      - ELASTIC_PASSWORD=changeme-use-strong-password
      - cluster.name=selfhosted-search
    ulimits:
      memlock:
        soft: -1
        hard: -1
    restart: unless-stopped

  kibana:
    image: docker.elastic.co/kibana/kibana:9.3.1
    container_name: kibana
    ports:
      - "5601:5601"
    environment:
      - ELASTICSEARCH_HOSTS=http://elasticsearch:9200
      - ELASTICSEARCH_USERNAME=kibana_system
      - ELASTICSEARCH_PASSWORD=changeme-kibana-password
    depends_on:
      - elasticsearch
    restart: unless-stopped

volumes:
  es_data:

Start the stack:

docker compose up -d

Initial Setup

Without Security

# Check cluster health
curl http://localhost:9200/_cluster/health?pretty

# Index a document
curl -X POST http://localhost:9200/my-index/_doc \
  -H "Content-Type: application/json" \
  -d '{"title": "Self-Hosting Guide", "content": "How to run your own services"}'

# Search
curl "http://localhost:9200/my-index/_search?q=self-hosting&pretty"

With Security

When security is enabled, Elasticsearch generates TLS certificates and passwords on first start. Check logs for the auto-generated elastic user password:

docker logs elasticsearch 2>&1 | grep "Password"

Configuration

Key Environment Variables

Variable	Default	Description
`discovery.type`		Set to `single-node` for single-node deployment
`ES_JAVA_OPTS`		JVM heap: `-Xms1g -Xmx1g` (set both equal)
`xpack.security.enabled`	`true` (9.x)	Enable/disable security
`ELASTIC_PASSWORD`	Auto-generated	Password for `elastic` user
`cluster.name`	`elasticsearch`	Cluster identifier
`node.name`	Auto-generated	Node identifier
`bootstrap.memory_lock`	`false`	Lock JVM memory to prevent swapping

JVM Heap Sizing

Set -Xms and -Xmx to the same value. Guidelines:

Minimum: 512m (testing only)
Small deployments: 1g-2g
Medium: 4g-8g
Large: 16g-31g (never exceed 31g — compressed oops threshold)
Rule of thumb: 50% of available RAM, max 32g

Advanced Configuration

Index Lifecycle Management (ILM)

Automatically manage index retention for logs:

curl -X PUT "http://localhost:9200/_ilm/policy/logs-policy" \
  -H "Content-Type: application/json" \
  -d '{
    "policy": {
      "phases": {
        "hot": {"actions": {"rollover": {"max_size": "10gb", "max_age": "7d"}}},
        "delete": {"min_age": "30d", "actions": {"delete": {}}}
      }
    }
  }'

Snapshot Repository (Backup)

# Register a filesystem snapshot repository
curl -X PUT "http://localhost:9200/_snapshot/my_backup" \
  -H "Content-Type: application/json" \
  -d '{"type": "fs", "settings": {"location": "/usr/share/elasticsearch/data/backups"}}'

# Create a snapshot
curl -X PUT "http://localhost:9200/_snapshot/my_backup/snapshot_1"

Upgrading from v8

Elasticsearch 9 dropped TLSv1.1, removed several deprecated settings, and returns 429 instead of 5xx for timeouts. For most single-node Docker deployments:

Take a snapshot before upgrading
Remove deprecated env vars if you added any: cluster.routing.allocation.disk.watermark.enable_for_single_data_node, xpack.searchable.snapshot.allocate_on_rolling_restart
Unfreeze frozen indices — v9 cannot read frozen indices
Update client code if you use the _knn_search endpoint (replaced by _search with knn query)
Update the image tag to 9.3.1 and Kibana to match

Most standard single-node setups with discovery.type=single-node work without config changes beyond the image tag.

Reverse Proxy

Configure your reverse proxy to forward to port 9200 (API) and 5601 (Kibana). See Reverse Proxy Setup.

Backup

Use Elasticsearch’s snapshot API for consistent backups. The data volume can also be backed up when the service is stopped. See Backup Strategy.

Troubleshooting

Container Crashes on Start

Symptom: Exit code 78, max virtual memory areas vm.max_map_count error. Fix: Run sudo sysctl -w vm.max_map_count=262144 on the host. This is the most common Elasticsearch Docker issue.

Out of Memory

Symptom: Container killed by OOM killer. Fix: Reduce ES_JAVA_OPTS heap size. Ensure the host has at least 2x the JVM heap size in total RAM (the OS needs RAM for file system cache). Set memlock ulimits.

Slow Queries

Symptom: Search responses take seconds. Fix: Check index mappings — use keyword for exact match fields, text for full-text search. Add SSD storage. Increase JVM heap. Check for expensive aggregations.

Cluster Status Yellow/Red

Symptom: _cluster/health returns yellow or red status. Fix: Yellow = unassigned replica shards (normal for single-node). Set number_of_replicas to 0 for single-node: curl -X PUT "localhost:9200/_settings" -H "Content-Type: application/json" -d '{"number_of_replicas": 0}'. Red = primary shard failures, check logs.

Resource Requirements

RAM: 2 GB minimum (1 GB JVM heap + OS), 4-8 GB recommended
CPU: Medium (indexing is CPU-intensive)
Disk: SSD recommended, size depends on data volume

Verdict

Elasticsearch is the industry standard for search and analytics. If you need full-text search, log aggregation, or an observability platform, nothing else has the same depth of features and ecosystem. The trade-off is significant resource requirements and complexity.

Choose Elasticsearch for search + analytics + logging. Choose OpenSearch for the same capabilities with a fully open-source license. Choose Meilisearch or Typesense if you just need application search without the analytics overhead.

Frequently Asked Questions

Is Elasticsearch still open source?

Elasticsearch moved from Apache 2.0 to a dual SSPL/Elastic License in 2021, then switched to AGPL-2.0 in 2024. The AGPL allows self-hosting and internal use without restrictions. If you need a permissively licensed alternative, OpenSearch is a community fork under Apache 2.0 with the same core search and analytics capabilities.

Why does Elasticsearch need so much RAM?

Elasticsearch runs on the JVM (Java Virtual Machine), which requires a dedicated heap for indexing and search operations. The operating system also needs RAM for filesystem caching, which dramatically improves search performance. The minimum practical heap is 1 GB, but 2-4 GB is recommended for useful workloads. Never set the heap above 50% of available RAM or above 31 GB.

What is vm.max_map_count and why is it required?

Elasticsearch uses memory-mapped files (mmap) extensively for efficient data access. The Linux default of 65536 mmap areas is too low. Setting vm.max_map_count=262144 on the host allows Elasticsearch to map enough memory for its indices. Without this setting, the container crashes on startup with exit code 78.

How does Elasticsearch compare to Meilisearch or Typesense?

Elasticsearch is a full search and analytics engine — it handles full-text search, aggregations, log analysis, and observability. Meilisearch and Typesense are purpose-built application search engines that are much lighter, faster to set up, and easier to use for basic search functionality. Choose Elasticsearch when you need analytics and log aggregation alongside search; choose Meilisearch or Typesense when you just need fast application search.

Can I run Elasticsearch on a Raspberry Pi?

Not recommended. Elasticsearch’s JVM requires a minimum of 2 GB RAM, and practical deployments need 4 GB+. A Raspberry Pi 4 with 4 GB or 8 GB can technically run a small instance, but performance will be poor. For ARM-based search, Meilisearch or Typesense are much more suitable — both run natively on ARM with far lower resource requirements.

What does “cluster status yellow” mean?

Yellow status means all primary shards are assigned but some replica shards are unassigned. On a single-node deployment, this is normal — replicas need a second node to exist on. Fix it by setting number_of_replicas to 0 for your indices. Red status means primary shard failures and requires investigation.