Tag Archives: output

Elasticsearch, REST

Elasticsearch REST Synchronous and Asynchronous performRequest APIs

5 Comments

Rest client can perform Synchronous and Asynchronous both type of requests. Synchronous Api’s return response with response code while Asynchronous api’s return response as void and accept extra argument extraResponseListener as callback which respond on completion and failure.

Synchronous  performRequest

Response performRequest(String method, String endpoint, Header... headers) throws IOException;

Response performRequest(String method, String endpoint, Map<String, String> params, Header... headers) throws IOException;

Response performRequest(String method, String endpoint, Map<String, String> params, HttpEntity entity, Header... headers) throws IOException;

Response performRequest(String method, String endpoint, Map<String, String> params, HttpEntity entity, HttpAsyncResponseConsumerFactory responseConsumerFactory, Header... headers) throws IOException;

Asynchronous  performRequest

void performRequestAsync(String method, String endpoint, ResponseListener responseListener, Header... headers);

void performRequestAsync(String method, String endpoint, Map<String, String> params, ResponseListener responseListener, Header... headers);

void performRequestAsync(String method, String endpoint, Map<String, String> params, HttpEntity entity, ResponseListener responseListener, Header... headers);

void performRequestAsync(String method, String endpoint, Map<String, String> params, HttpEntity entity, HttpAsyncResponseConsumerFactory responseConsumerFactory, ResponseListener responseListener, Header... headers);

Details about Parameters:

Method :  Elasticsearch support all rest opration like GET, POST,PUT,DELETE.
Endpoint: Url to call elasticsearch apis like (/_cat/indices for getting list of indexes on elasticsearch).
Params: This field is optional and will pass as query strings parameter.
Entity: This field is optional and will pass in method type like POST,PUT or filter query request.
Headers: This is optional will pass if request need header param.

Additional Parameter for Asynchronous call:

ResponseConsumerFactory: Optional and will use to create HttpAsynchResponseConsumer for callback response for request.

ResponseListener: Listener return callback response as request was complete successfully or failure.

Read More on Elasticsearch REST

Integration

Integrate Filebeat, Kafka, Logstash, Elasticsearch and Kibana

ELK, Filebeat, Kafka

Filebeat and Kafka Integration

6 Comments

Kafka can consume messages published by Filebeat based on configuration  filebeat.yml file for Kafka Output.

Filebeat Kafka Output Configuration

Filebeat.yml  required below fields to connect and publish message to Kafka for configured topic. Kafka will create Topics dynamically based on filebeat requirement.

output.kafka:
#The list of Kafka broker addresses from where to fetch the cluster metadata.
#The cluster metadata contain the actual Kafka brokers events are published to.
hosts: <strong>["localhost:9092"]</strong>

# The Kafka topic used for produced events. The setting can be a format string
topic: <strong>Topic-Name</strong>

# Authentication details. Password is required if username is set.
#username: ''
#password: ''

For more information about filebeat Kafka Output  configuration option refers below Links.

Let me know your thought on this post.

Happy Learning !!!

Read More on Kafka

Integration

Integrate Filebeat, Kafka, Logstash, Elasticsearch and Kibana