Filebeat is a light weight agent on server for shipping & forwarding the logs, Filebeat can monitors log files & directories changes and forward log lines to different target Systems like Logstash, Kafka ,Elasticsearch or files etc. Filebeat play a very important role in centralize logging where files logs from multiple system are forwarded to centralize system for parsing and monitoring for analysis.

Filebeat work like tail command in Unix/Linux.

Latest Filebeat Version :   8.8.2

Why Filebeat ?

Filebeat is so popular in terms of Centralize Logging with ELK (Elasticsearch, Logstash and Kibana) by following reasons:

• Lightweight agent for shipping logs.
• Forward and centralize files and logs.
• Robust (Not miss a single beat)

How Filebeat Work?

Filebeat starts prospectors to locate corresponding to each log file path mentioned in filebeat configuration file. Filebeat start a periodic harvester, which identify changes on file based on inode value, do tail to read change logs and send it to spooler to aggregate it. Processors (If configure) will perform different operation based on condition in spooler. Spooler send this aggregated data to target Systems like Logstash, Kafka, Elasticsearch or files etc.

In the below diagram you can see for each file reading, Filebeat create the prospectors once it watch any change in files harvester take these changes and forward to configured output system (Elasticsearch, Logstash, redis or Kafka etc.)

Filebeat Installation

You can download and install filebeat by following link : Filebeat Download

See Also

• Complete Integration Example Filebeat, Kafka, Logstash, Elasticsearch and Kibana
• Filebeat Multiline Configuration Changes for Object, StackTrace and XML
• Filebeat and Kafka Integration
• Filebeat Integration with Logstah
• Filebeat Issues & Solutions
• Log4j2: How to Mask Logs Personal/Confidential/SPI Information