Multiline configuration is required if need to handle multilines on filebeat server end. That will help for logs type like stackTrace for exception, print objects, XML, JSON etc. where standard log4j format does’t work so this type of lines can be combined with previous line where log4j format was applied.
Below are filebeat configuration for multiline.
multiline.pattern: The regexp Pattern that has to be matched. The example pattern matches all lines starting with [DEBUG,ALERT,TRACE,WARNING log level that can be customize according to your logs line format. But that is generic one that will help most of cases.
multiline.pattern: ‘^[([Aa]lert|ALERT|[Tt]race|TRACE|[Dd]ebug|DEBUG|[Nn]otice|NOTICE|[Ii]nfo|INFO|[Ww]arn?(?:ing)?|WARN?(?:ING)?|[Ee]rr?(?:or)?|ERR?(?:OR)?|[Cc]rit?(?:ical)?|CRIT?(?:ICAL)?|[Ff]atal|FATAL|[Ss]evere|SEVERE|EMERG(?:ENCY)?|[Ee]merg(?:ency)?)’
Default is false for negate. Defines if the pattern match should be negated or not.
multiline.negate: true
multiline.match define if pattern not match with above pattern where these line need to append. Possible values are “after” or “before”.
multiline.match: after
If you will set this max line after these number of multiline all will ignore
multiline.max_lines: 50
For Example :
multiline.pattern: ‘^\[([Aa]lert|ALERT|[Tt]race|TRACE|[Dd]ebug|DEBUG|[Nn]otice|NOTICE|[Ii]nfo|INFO|[Ww]arn?(?:ing)?|WARN?(?:ING)?|[Ee]rr?(?:or)?|ERR?(?:OR)?|[Cc]rit?(?:ical)?|CRIT?(?:ICAL)?|[Ff]atal|FATAL|[Ss]evere|SEVERE|EMERG(?:ENCY)?|[Ee]merg(?:ency)?)’
multiline.negate: true
multiline.match: after
#multiline.max_lines: 50
Sample Configuration file for multiline configuration.
Sample filebeat.yml file for Prospectors,Multiline and Logging Configuration
Integration
Complete Integration Example Filebeat, Kafka, Logstash, Elasticsearch and Kibana
Read More
To read more on Filebeat topics, sample configuration files and integration with other systems with example follow link Filebeat Tutorial and Filebeat Issues. To know more about YAML follow link as YAML Tutorial.
Leave you feedback to enhance more on this topic so that make it more helpful for others.
Your Feedback Motivate Us
If our FacingIssuesOnIT Experts solutions guide you to resolve your issues and improve your knowledge. Please share your comments, like and subscribe to get notifications for our posts.
Happy Learning !!!
13 thoughts on “Filebeat Multiline Configuration Changes for Object, StackTrace and XML”