Multiline configuration is required if need to handle multilines on filebeat server end. That will help for logs type like stackTrace for exception, print objects, XML, JSON etc. where standard log4j format does’t work so this type of lines can be combined with previous line where log4j format was applied.
Below are filebeat configuration for multiline.
multiline.pattern: The regexp Pattern that has to be matched. The example pattern matches all lines starting with [DEBUG,ALERT,TRACE,WARNING log level that can be customize according to your logs line format. But that is generic one that will help most of cases.
Default is false for negate. Defines if the pattern match should be negated or not.
multiline.match define if pattern not match with above pattern where these line need to append. Possible values are “after” or “before”.
If you will set this max line after these number of multiline all will ignore
For Example :
multiline.pattern: ‘^\[([Aa]lert|ALERT|[Tt]race|TRACE|[Dd]ebug|DEBUG|[Nn]otice|NOTICE|[Ii]nfo|INFO|[Ww]arn?(?:ing)?|WARN?(?:ING)?|[Ee]rr?(?:or)?|ERR?(?:OR)?|[Cc]rit?(?:ical)?|CRIT?(?:ICAL)?|[Ff]atal|FATAL|[Ss]evere|SEVERE|EMERG(?:ENCY)?|[Ee]merg(?:ency)?)’ multiline.negate: true multiline.match: after #multiline.max_lines: 50
Sample Configuration file for multiline configuration.
Sample filebeat.yml file for Prospectors,Multiline and Logging Configuration
Complete Integration Example Filebeat, Kafka, Logstash, Elasticsearch and Kibana
To read more on Filebeat topics, sample configuration files and integration with other systems with example follow link Filebeat Tutorial and Filebeat Issues. To know more about YAML follow link as YAML Tutorial.
Leave you feedback to enhance more on this topic so that make it more helpful for others.
13 thoughts on “Filebeat Multiline Configuration Changes for Object, StackTrace and XML”