Category "logstash"

Upserting documents in elasticsearch using multiple fields instead of one

Further to question Upsert documents in Elasticsearch using custom ID field, now I need to upsert the documents with 2+ fields from the input. Taking the same e

Send data to Logstash in docker-compose environment

I've setup the ELK stack (Elasticsearch, Logstash and Kibana) by following this Github repo. Now I want to send data to logstash, for example specifying a pipel

Elasticsearch&Logstash Log4j Vulnerabilities

We have log4j vulnerabilities for Elasticsearch and Logstash in the following paths: Path : /usr/share/Elasticsearch/lib/log4j-core-2.11.1.jar Path : /usr/share

How to detect logstash input connection error

How can I monitor and detect errors when connecting kafka to logstash. Say for example my kafka broker is down and no connection is established between kafka an

ELK stack error elastic search don't authorize Logstash

I followed up this blog to start ELK stack from docker compose file but used version 8.1.2. It is not running successfully elastic search don't authorize Logst

How to test logstash Marker in Junit

I have to create a Junit test for my class logging Map<String, Object> mapMarker = new HashMap<>(); mapMarker.put("input_source", "test_input_source

Can Kafka Connect consume data from a separate kerberized Kafka instance and then route to Splunk?

My pipeline is: Kerberized Kafka --> Logstash (hosted on a different server) --> Splunk. Can I replace the Logstash component with Kafka Connect? Could

logstash settings are invalid

I'am trying to start logstash on windows with a custom conf file but I get always the same error : Your settings are invalid. Reason: Setting "" doesn't exist.

Logstash with persistent queue

I have started logstash using following configurations: Inside logstash.yml: queue.type: persisted queue.max_bytes: 8gb queue.checkpoint.writes: 1 configura

elasticsearch es_rejected_execution_exception

I'm trying to index a 12mb log file which has 50,000 logs. After Indexing around 30,000 logs, I'm getting the following error [2018-04-17T05:52:48,254][INFO ]

Is it possible to configure multiple output for a filebeat?

In one of our applications we parse the application logs using logstash and indexing them into elasticsearch. Our simple architecture is logfiles ---> filebe

which jdbc (connector/j) to install for mac os

I'm trying to download jdbc connector, but I cannot find mac os from the selection options from the link below: https://dev.mysql.com/downloads/connector/j/ W

Convert Ticks to @timestamp in logstash with Ruby-plugin

I query every minute against a MSSQL Database with jdbc-plugin. In this Database my timestamp is stored in ticks. Field-name is lastupdate. Now I wanted to conv

Logstash KV filter help needed

I am trying to write a logstash configuration for the following log entry. 102.89.98.123,-,"[04/Jun/2018:12:36:44 -0400]",?q= (AggrDept:John%20OR%20AggrBeept:

How to watch the logstash log?

For my enterprise application distributed and structured logging, I use logstash for log aggregation and elastic search as log storage. I have the clear control