Tag Archives: ELK

How to Configure Filebeat, Kafka, Logstash Input , Elasticsearch Output and Kibana Dashboard


How to Configure and integrate Filebeat, Kafka, Logstash Input , Elasticsearch Output and Kibana Dashboard Continue reading

Advertisements
Posted in Elasticsearch, ELK, Example, Filebeat, JSON, Kafka, Kibana, Logstash, Zookeeper | Tagged , , , , , , , ,

Integrate Filebeat, Kafka, Logstash, Elasticsearch and Kibana


Integrate Filebeat, Kafka,Logstash,Elasticsearch and Kibana for Organizantion level for log analysis. Continue reading

Posted in Elasticsearch, Filebeat, JSON, Kafka, Kibana, Logstash, Zookeeper | Tagged , , , , , , , , | 29 Comments

Logstash Input and Output to/from Kafka Example


Logstash Input and Output to/from Kafka Example Continue reading

Posted in Kafka, Logstash | Tagged , , , , , | 6 Comments

Integrate Java with Kafka


Below examples are for Kafka Logs Producer and Consumer by Kafka Java API. Where Producer is sending logs from file to Topic1 on Kafka server and same logs Consumer is subscribing from Topic1. While Kafka Consumer can subscribe logs from … Continue reading

Posted in Kafka | Tagged , , , , , | 6 Comments

Setup Kafka Cluster for Single Server/Broker


For setting up Kafka Cluster for Single Broker . Follow below steps : Download and Installation Download Latest version of Kafka from link download , copy it to installation directory and run below command to install it. Configuration Changes for Zookeeper … Continue reading

Posted in Kafka | Tagged , , , , , , , | 7 Comments

Kafka Introduction and Architecture


Kafka is Open source distributed, Steam Processing, Message Broker platform written in Java and Scala developed by Apache Software Foundation. Kafka is massively use for enterprise infrastructure to process stream data or transaction logs on real time. Kafka provide unified, … Continue reading

Posted in Kafka | Tagged , , , , , , , , , | 7 Comments

Logstash Custom Grok Pattern


Logstash provide some predefined grok pattern for some standard cases like URL , INT, GREEDYDATA, WORD etc. We can customize and define our own grok pattern also. Why do we need customize Grok Pattern? If our requirement is define our … Continue reading

Posted in Logstash | Tagged , ,