Elasticsearch “high disk watermark exceeded on one or more nodes”


This issue happen on Elasticsearch anytime   because your diskspace storage reached to more than 85% because in elasticsearch by default watermark is 85% of disk storage after reaching this limit elasticsearch stop working and print below warning in logs.

[2017-05-12T16:52:48,239][WARN ][o.e.c.r.a.DiskThresholdMonitor] [r78TO-I] high disk watermark [90%] exceeded on [r78TO-IgT9yPQQUgILl_5A][r78TO-I][C:\
Users\Facing Issues on IT\elasticsearch-5.4.0\data\nodes\0] free: 11.7gb[9.8%], shards will be relocat
ed away from this node
[2017-05-12T16:52:48,242][INFO ][o.e.c.r.a.DiskThresholdMonitor] [r78TO-I] rerouting shards: [high disk watermark exceeded on one or more nodes]
[2017-05-12T16:53:18,297][WARN ][o.e.c.r.a.DiskThresholdMonitor] [r78TO-I] high disk watermark [90%] exceeded on [r78TO-IgT9yPQQUgILl_5A][r78TO-I][C:\
Users\Facing Issues on IT\elasticsearch-5.4.0\data\nodes\0] free: 11.7gb[9.8%], shards will be relocat
ed away from this node
[2017-05-12T16:53:48,306][WARN ][o.e.c.r.a.DiskThresholdMonitor] [r78TO-I] high disk watermark [90%] exceeded on [r78TO-IgT9yPQQUgILl_5A][r78TO-I][C:\
Users\Facing Issues on IT\elasticsearch-5.4.0\data\nodes\0] free: 11.7gb[9.8%], shards will be relocat
ed away from this node

Solution :

We have to set below properties in config/elasticsearch.yml file for threshold value for disk storage , low and high watermark value.

cluster.routing.allocation.disk.threshold_enabled: true
cluster.routing.allocation.disk.watermark.low: 5gb
cluster.routing.allocation.disk.watermark.high: 2gb

Issues Solution

For more Elasticsearch issues solution follow link Common Elasticsearch Issues.

About Saurabh Gupta

My Name is Saurabh Gupta, I have approx. 10 Year of experience in Information Technology World manly in Java/J2EE. During this time I have worked with multiple organization with different client, so many technology, frameworks etc.
This entry was posted in Uncategorized. Bookmark the permalink.