Integrate Filebeat with Kafka

Kafka can consume messages published by Filebeat based on configuration  filebeat.yml file for Kafka Output.

Filebeat Kafka Output Configuration

Filebeat.yml  required below fields to connect and publish message to Kafka for configured topic. Kafka will create Topics dynamically based on filebeat requirement.

#The list of Kafka broker addresses from where to fetch the cluster metadata.
#The cluster metadata contain the actual Kafka brokers events are published to.
hosts: <strong>["localhost:9092"]</strong>

# The Kafka topic used for produced events. The setting can be a format string
topic: <strong>Topic-Name</strong>

# Authentication details. Password is required if username is set.
#username: ''
#password: ''

For more information about filebeat Kafka Output  configuration option refers below Links.

Read More on Kafka


Integrate Filebeat, Kafka, Logstash, Elasticsearch and Kibana


About Saurabh Gupta

My Name is Saurabh Gupta, I have approx. 11 Year of experience in Information Technology World manly in Java/J2EE. During this time I have worked with multiple organization with different client, so many technology, frameworks etc.
This entry was posted in Filebeat, Kafka and tagged , , , , , , . Bookmark the permalink.

6 Responses to Integrate Filebeat with Kafka

  1. Pingback: Setup Kafka Cluster for Multi/Distributed Servers/Brokers | Facing Issues On IT

  2. Pingback: Setup Kafka Cluster for Single Server/Broker | Facing Issues On IT

  3. Pingback: Kafka Introduction and Architecture | Facing Issues On IT

  4. Pingback: Integrate Java with Kafka | Facing Issues On IT

  5. Pingback: Kafka Server Properties Configuration | Facing Issues On IT

  6. Pingback: Integrate Logstash with Kafka | Facing Issues On IT

Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s