elastic search cheat sheet

Elastic Search Cheat Sheet with Kibana REST & curl commands

First Set ELASTICSEARCH URL

ELASTICSEARCH_URL="<cluster url with protocol and optinally port number>"

Elastic Search Administration

Get the health of the cluster

GET /_cluster/health

curl -s -XGET "$ELASTICSEARCH_URL/_cluster/health?pretty"

Get cluster stats

curl -s -XGET "$ELASTICSEARCH_URL/_cluster/stats?pretty&human=true" 

GET /_cluster/stats?human=true

Get the cluster state

GET /_cluster/state

curl -s -XGET "$ELASTICSEARCH_URL/_cluster/state?pretty"

Cluster Information

GET /

curl -s -XGET $ELASTICSEARCH_URL

Get Size of all indices

curl -s -XGET "$ELASTICSEARCH_URL/_cluster/stats?pretty&human=true" | jq -r '.indices.store.size'

Get Size of Elastic Search data on each node vs node size

GET /_cat/allocation?v 

curl -s -XGET "$ELASTICSEARCH_URL/_cat/allocation?v"

Get Node information

GET /_cat/nodes?v 

curl -s -XGET "$ELASTICSEARCH_URL/_cat/nodes?v"

Get Thread Pool

GET /_cat/thread_pool?v&s=active:desc

curl -s -XGET "$ELASTICSEARCH_URL/_cat/thread_pool?v&s=active:desc"

Get total number of shards

GET _cluster/stats?filter_path=indices.shards.total

curl -s -XGET "$ELASTICSEARCH_URL/_cluster/stats?filter_path=indices.shards.total&pretty"

curl -s -XGET "$ELASTICSEARCH_URL/_cat/shards" | wc -l

Get total number of indices

curl "$ELASTICSEARCH_URL/_cat/indices?h=index" | wc -l

Get List of Pending Tasks

GET /_cluster/pending_tasks

curl -s -XGET "$ELASTICSEARCH_URL/_cluster/pending_tasks?pretty"

Get List of All mappings

GET /_mapping

curl -s -XGET "$ELASTICSEARCH_URL/_mapping?pretty"

Get all plugins

GET /_cat/plugins?v&s=component 

curl -s -XGET "$ELASTICSEARCH_URL/_cat/plugins?v&s=component"

Total Free Space in the CLuster

curl -s -XGET "$ELASTICSEARCH_URL/_cluster/stats?pretty&human=true" | jq -r '.nodes.fs.free'

Format Elastic Search Output; Examples

GET /_cluster/health?format=yaml

curl -s -XGET "$ELASTICSEARCH_URL/_cluster/health?format=yaml"

GET /_cluster/health?format=csv

curl -s -XGET "$ELASTICSEARCH_URL/_cluster/health?format=csv"

GET /_cluster/health?format=txt

curl -s -XGET "$ELASTICSEARCH_URL/_cluster/health?format=txt"

GET /_cluster/health?format=json

curl -s -XGET "$ELASTICSEARCH_URL/_cluster/health?format=json&pretty"

Elastic Search DataBase Operations

Set Index Name

INDEX_NAME="<index name>"

Create an index (default)

PUT /<index-name> 

curl -X PUT "$ELASTICSEARCH_URL/$INDEX_NAME"

Create an index with mappings and settings

PUT /index-name
{
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 1
  },
  "mappings": {
    "properties": {
      "name": {
        "type": "text"
      },
      "price": {
        "type": "integer"
      }
    }
  }
}

curl -X PUT "$ELASTICSEARCH_URL/$INDEX_NAME" -H "Content-Type: application/json" -d '{
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 1
  },
  "mappings": {
    "properties": {
      "name": {
        "type": "text"
      },
      "price": {
        "type": "integer"
      }
    }
  }
}'

Add a document

POST /<index-name>/_doc
{
  "name": "Shivam",
  "age": 24
}

curl -X POST "$ELASTICSEARCH_URL/$INDEX_NAME/_doc" -H "Content-Type: application/json" -d '{
  "name": "Shivam",
  "age": 24
}'

Get index mappings and settings

GET /<index-name> 

curl -s -XGET "$ELASTICSEARCH_URL/$INDEX_NAME?pretty"

Empty an index

POST /<index-name>/_delete_by_query?q=* 

curl -XPOST "$ELASTICSEARCH_URL/$INDEX_NAME/_delete_by_query?q=*"

Delete an index

DELETE /<index-name> 

curl -X DELETE "$ELASTICSEARCH_URL/$INDEX_NAME"

List All Indices (Most useful), Sort by descending order of size, show header and important columns only

GET /_cat/indices?v&s=store.size:desc&h=index,pri,rep,docs.count,docs.deleted,store.size,pri.store.size 

curl -s -XGET "$ELASTICSEARCH_URL/_cat/indices?v&s=store.size:desc&h=index,pri,rep,docs.count,docs.deleted,store.size,pri.store.size"

List All Indices with metadata

GET /_cat/indices?v

curl -s -XGET "$ELASTICSEARCH_URL/_cat/indices?v"

Get All Indices list only

GET /_cat/indices?h=index

curl -s -XGET $ELASTICSEARCH_URL/_cat/indices?h=index

Get All Indices sorted in descending order of size

GET /_cat/indices?v&s=store.size:desc

curl -s -XGET "$ELASTICSEARCH_URL/_cat/indices?v&s=store.size:desc"

Get List of All aliases , Sort by index name

GET /_cat/aliases?v&s=index 

curl -s -XGET "$ELASTICSEARCH_URL/_cat/aliases?v&s=index"

Get List of All aliases

GET /_cat/aliases?v 

curl -s -XGET "$ELASTICSEARCH_URL/_cat/aliases?v"

Get all data inside the index

GET /<index-name>/_search?pretty
{
  "query": {
    "match_all": {}
  }
}

curl -s -XGET "$ELASTICSEARCH_URL/$INDEX_NAME/_search?pretty" -H 'Content-Type: application/json' -d'
{
    "query": {
        "match_all": {}
    }
}'

Update an entire doc by it ID

PUT /<index-name>/_doc/<doc-id>
{
  "name": "Anand",
  "age": 25
}

curl -X PUT "$ELASTICSEARCH_URL/$INDEX_NAME/_doc/$DOC_ID" -H "Content-Type: application/json" -d '{
  "name": "Anand",
  "age": 25
}'

Update a specific field document in an index by its id

POST /<index-name>/_update/<doc-id>
{
  "doc": {
    "age": 99
  }
}

DOC_ID="<Get document id>"
curl -X POST "$ELASTICSEARCH_URL/$INDEX_NAME/_update/$DOC_ID" -H "Content-Type: application/json" -d '{
  "doc": {
    "age": 99
  }
}'

Update all document in an index by script

POST /<index-name>/_update_by_query
{
  "script": {
    "source": "ctx._source.age += params.count",
    "lang": "painless",
    "params": {
      "count": 10
    }
  }
}

curl -X POST "$ELASTICSEARCH_URL/$INDEX_NAME/_update_by_query" -H 'Content-Type: application/json' -d'
{
  "script" : {
    "source": "ctx._source.age += params.count",
    "lang": "painless",
    "params" : {
      "count" : 10
    }
  }
}'

Bulk Operation using bulk API

POST /<index-name>/_update_by_query
{
  "script": {
    "source": "ctx._source.age += params.count",
    "lang": "painless",
    "params": {
      "count": 10
    }
  }
}

curl -XPOST "$ELASTICSEARCH_URL/$INDEX_NAME/_bulk?pretty" -H "Content-Type: application/json" -d '
{"index":{"_id":"1"}
{"name": "Ramu","age": 45}
{"index":{"_id":"2"}
{"name": "Lemon","age": 1}
{"delete":{"_id":"1"}}
{"update":{"_id":"2"}}
{"doc":{"name":"Orange"}}
'

Get document count in an index

GET /<index-name>/_count 

curl -s -XGET "$ELASTICSEARCH_URL/$INDEX_NAME/_count?pretty"

Make an index read-only

Block write to the source index / make the index read only

PUT /<index-name>/_settings?pretty
{
  "index.blocks.write": true
}

curl -X PUT "$ELASTICSEARCH_URL/$INDEX_NAME/_settings?pretty" -H 'Content-Type: application/json' -d '
{
   "index.blocks.write": true
}'

Verify write is blocked

GET /<index-name>/_settings 

curl -s -XGET "$ELASTICSEARCH_URL/$INDEX_NAME/_settings?pretty"

Undo block write / Undo read-only mode

PUT /<source-index>/_settings
{
  "index.blocks.write": false
}

curl -X PUT "$ELASTICSEARCH_URL/$SOURCE_INDEX/_settings" -H 'Content-Type: application/json' -d '
{
   "index.blocks.write": false
}'

Backup an index

SOURCE_INDEX="<copy from index>"  

DESTINATION_INDEX="<copy to index>"

Mandatory step to block write to the source index / make the index read only

PUT /<source-index>/_settings?pretty
{
  "index.blocks.write": true
}

curl -X PUT "$ELASTICSEARCH_URL/$SOURCE_INDEX/_settings?pretty" -H 'Content-Type: application/json' -d '
{
   "index.blocks.write": true
}'

Verify write is blocked

GET /<source-index>/_settings 

curl -s -XGET "$ELASTICSEARCH_URL/$SOURCE_INDEX/_settings?pretty"

Copy data from source to destination

POST /<source-index>/_clone/<destination-index>?pretty

curl -XPOST "$ELASTICSEARCH_URL/$SOURCE_INDEX/_clone/$DESTINATION_INDEX?pretty" 

Verify data is copied to destination

POST /<index-name>/_search?pretty
{
  "query": {
    "match_all": {}
  }
}

curl -s -XGET "$ELASTICSEARCH_URL/$DESTINATION_INDEX/_search?pretty" -H 'Content-Type: application/json' -d'
{
    "query": {
        "match_all": {}
    }
}'

Undo block write / Undo read-only mode

PUT /<source-index>/_settings
{
  "index.blocks.write": false
}

curl -X PUT "$ELASTICSEARCH_URL/$SOURCE_INDEX/_settings" -H 'Content-Type: application/json' -d '
{
   "index.blocks.write": false
}'

Open and close indexes to save memory and CPU

close: A closed index is essentially frozen. Search and write operations are blocked.

POST /<index-name>/_close 

curl -XPOST "$ELASTICSEARCH_URL/$INDEX_NAME/_close"

open: An open index is available for search and write operations. This is the typical state for an index you're actively using for data storage and retrieval.

POST /<index-name>/_open 

curl -XPOST "$ELASTICSEARCH_URL/$INDEX_NAME/_open"

Get Index Stats

GET /<index-name>/_stats 

curl -s -XGET "$ELASTICSEARCH_URL/$INDEX_NAME/_stats?pretty"

Get Index Segments Information

GET /<index-name>/_segments 

curl -s -XGET "$ELASTICSEARCH_URL/$INDEX_NAME/_segments?pretty"

Get Index Recovery Status (Human Readable)

GET /<index-name>/_recovery&human 

curl -s -XGET "$ELASTICSEARCH_URL/$INDEX_NAME/_recovery?pretty&human"

Clear Index Cache

POST /<index-name>/_cache/clear 

curl -XPOST "$ELASTICSEARCH_URL/$INDEX_NAME/_cache/clear"

Refresh Index

POST /<index-name>/_refresh 

curl -XPOST "$ELASTICSEARCH_URL/$INDEX_NAME/_refresh"

Flush Index

POST /<index-name>/_flush 

curl -XPOST "$ELASTICSEARCH_URL/$INDEX_NAME/_flush"

Force Merge Index Segments

POST /<index-name>/_forcemerge 

curl -XPOST "$ELASTICSEARCH_URL/$INDEX_NAME/_forcemerge"