Environment and Commandline arguments setting to filebeat.yml

We can use environment variables and arguments from command line  references in the filebeat.yml file to set values or fields that need to be configurable during deployment. To configure fields and values use like this:

${VAR1}

Where VAR1 is the name of the environment variable or passing values from command line.

These configured values will replaced on start up by the value of environment variable or command line arguments. These configured fields are case sensitive if value not found will fail on parsing so better configure values with default value as given below.

${VAR1:default_value}

We pass command line arguments to set configured values with option -E with argument name.

We have configured in below file variable fields for server, kafkaHost and topic in below sample configuration file that way we can set other fields also as per requirement.

Use below command to run this sample configuration file:

./filebeat -c filebeat.yml -d publish -E server=server1 -E kafkaHost=IP:PORT -E topicName=QC-TEST

For example above commandline arguments will pass to filebeat.yml file for below settings.

Prospectors Settings :

filebeat.prospectors:
- input_type: log
paths:
- /opt/app/${server}/logs/app1-debug*.log*
fields_under_root: true

Kafka Output Settings:

output.kafka:
#enabled: true
hosts: ["${kafkaHost}"]
topic: ${topicName}

Integration

Complete Integration Example Filebeat, Kafka, Logstash, Elasticsearch and Kibana

Read More

To read more on Filebeat topics, sample configuration files and integration with other systems with example follow link Filebeat Tutorial  and  Filebeat Issues.To Know more about YAML follow link YAML Tutorials.

Leave you feedback to enhance more on this topic so that make it more helpful for others.