Important Operating System Configurations For Elasticsearch

Hi Everyone. I will talk about important elasticsearch system configurations in this article. Elasticsearch requires to some OS – server configurations changes to ensure optimal performance and smooth operation. If you apply these requirements properly, you can enhance elasticsearch performance, reliability and scalability. Specially you must check these configurations when you go to production environment.

As I mentioned before, elasticsearch assumes that you are going to production when the network.host parameters change in elasticsearch.yml file.

ulimit 

ulimit settings define the maximum number of OS resources that a process or user can use. Elasticsearch works with large number of file. So elasticsearch needs to more open files descriptors than default value. You can change open files limit by editing /etc/security/limits.conf . Recommended value for elasticsearch user’s open file limit is 65535. To set this value add the following line to /etc/security/limits.conf

elasticsearch  -nofile  65535

If you want to check this config in elasticsearch, you can use Nodes Stats API.
GET _nodes/stats/process?filter_path=**.max_file_descriptors

LimitMEMLOCK

Locked memory refers to area of memory that is kept in RAM by an application. Locked memory is locked in physical RAM. In elasticsearch, locked memory refers to memory area outside of heap area. Elasticsearch uses this area to optimize some important operations. These operations can be indexing, searching and querying. Elasticsearch can improve faster access and performance. To set this limit add the following lines in your /usr/lib/systemd/system/elasticsearch.service file.

[Service]
LimitMEMLOCK=infinity

And reload systemd unit
systemctl daemon-reload

Swap Usage

Swapping is a memory management strategy in OS. It moves inactive memory blocks to disk and increase free space in RAM. Swapping can lead to performance degradation. In the context of Elasticsearch, swapping is very bad for performance and node stability. So you must disable the swapping on your server that elasticsearch running on. Or you can add the following lines in elasticsearch.yml

bootstrap.memory_lock: true

Virtual Memory

Elasticsearch uses mmapfs directory to store indices. Mmapfs refers to memory-mapped file system. It leverages memory-mapping feautre provided by OS. It maps index files to directly into memory. So you should increase the configuration limit about mmapfs usage in OS. You can use following lines to set this value in your /etc/sysctl.conf file.

vm.max_map_count=262144

Number of Threads

nproc refers to maximum number of processes on Linux. It determines the maximum number of thread that can be executed at same time. In the context of elasticsearch, it is critical to create new process whenever needed. You can set nproc value to 4096 in your limits.conf file. And systemd configure it automatically.

Ensure TMP directory permits executables

noexec option is a setting an OS that prevents files on disks from being executable. This setting is used for security reasons. Elastisearch uses libffi library for executing platform-dependent code. libffi libraries extract to tmp directory on OS. So this tmp directory must be executable. By default this library extract to /tmp directory on Linux. If your tmp directory is noexec, you need to remove this option or change -Djava.io.tmpdir in your jvm.option configuration files that stores ES_HOME directory. Or you can change ES_TMPDIR in your environment.

Lastly, there is a recommendation about TCP retransmission timeout. You must set this timeout value to 5. So cluster can quickly notice the node failure.

Thank you for your time. You can follow my youtube channel from here and you can read other elasticsearch posts from here.