Questions tagged [filebeat]
The filebeat tag has no usage guidance.
28
questions
5
votes
4
answers
50k
views
How to see if filebeat data is being sent to logstash
When I open up Kibana interface, I get an error to configure index when logstash-* is entered as a query:
kibana error: please specify a default index pattern
How can I see if filebeat is sending ...
- 171
3
votes
1
answer
4k
views
Why is this exclude_lines in filebeat excluding all logs?
I'm using ELK Stack, and I've got it working pretty well for most of my servers. The exception is that I have a gitlab server that has a ping to/from a gitlab-ci server that happens in the gitlab-...
- 1,136
3
votes
2
answers
3k
views
Different extractors for the same Graylog input?
I'm using Graylog's sidecar functionality with Filebeat to pickup a number of different log files off my server, including Syslog, Nginx and Java App. All of these flow into the same Graylog input for ...
- 632
2
votes
2
answers
881
views
systemd file not pickuping environment values
Systemd file for filebeat doesn't pickup env variables and throw as below
ExecStart=/usr/share/filebeat/bin/filebeat -environment systemd $BEAT_LOG_OPTS $BEAT_CONFIG_OPTS $BEAT_PATH_OPTS (code=exited, ...
2
votes
1
answer
15k
views
Filebeat can't connect to logstash on another server
Filebeat (11.11.11.11) can't connect to logstash (22.22.22.22) on another server (connection reset by peer). But filebeat services from other servers can do it.
Also I can connect from this server(11....
- 179
2
votes
1
answer
292
views
ELK logstash and core grok patterns
I'm evaluating the ELK stack with filebeat & logstash across a diverse range of applications/ servers.
I understand the power of customising my own grok patterns for each application/log, but to ...
- 141
2
votes
0
answers
71
views
Error while enrolling: empty access_token
I have successfully installed a full ELK Docker stack including Filebeat.
When I want to enroll a Filebeat instance, I get the following error:
Error while enrolling: empty access_token
According to ...
- 1,799
2
votes
0
answers
4k
views
Parsing JSON event in Logstash
I have log in following format, it is a plain json with nested fields.
{
"level": "info",
"message": {
"req": {
"headers": {
"host": "localhost:8080",
...
- 21
2
votes
0
answers
1k
views
filebeat makes a lot of I/O
We have filebeat on few servers that is writeing to elasticsearch. We can see that it is doing a lot of writes:
PID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND
353 be/3 ...
- 121
1
vote
1
answer
15k
views
cannot validate certificate - doesn't contain any IP SAN
I am currently in the process of installing ELK ( ElastricSearch, LogStash & Kibana) stack.
My ELK server IP address is 172.29.225.32.
Elastic Search config is ::
# -----------------------------...
- 185
1
vote
0
answers
70
views
Certificate only works from client side, how do I debug something like this? (graylog/filebeat/JVM keystore)
I created 2 key-certificate pairs with the exact same method. However, while trying to setup TLS on my graylog server to a remote filebeat node, it does not successfully connect when trying to connect ...
- 23
1
vote
0
answers
76
views
Suricata / Filebeat / ELK - iptables tee - Create virtual hosts
I have an IDS setup as follow:
Hardware / interfaces
WAN <----(brwan)> ROUTER / AP <(br0)----> LAN
\
-----(eth1)>...
1
vote
0
answers
374
views
Why does syslog create a user.log instead using syslog.log?
I have experienced something a bit weird for me. I have filebeat monitoring my rsyslog (syslog.log) file and sending it to my logstash.
I have noticed that after restarting filebeat where syslog is ...
- 193
1
vote
2
answers
3k
views
Kibana @timestamp mapping & filter
I'm using following system/package:
$ cat /etc/redhat-release
CentOS Linux release 7.2.1511 (Core)
$ rpm -q filebeat
filebeat-1.3.0-1.x86_64
$
with /etc/filebeat/filebeat.yml:
$ cat /etc/...
- 13.5k
0
votes
1
answer
313
views
Stop filebeat sending copius metadata
I am sending data from local log files with filebeat to graylog and I am getting a 20x storage overhead compared to the original files. There are a large amount of metadata fields however I can't seem ...
- 23
0
votes
1
answer
961
views
Grok filter is not working properly
I have Filebeat-7.1 installed in a Debian server, this Filebeat send data from files in this Debian server to server with Logstash 7.6 , here are the files config
Filebeat.yml:
#=====================...
- 1
0
votes
1
answer
437
views
Can't find docker log files for Filebeat
I'm trying to aggregate logs from my Kubernetes cluster into Elasticsearch server.
To do that, I've deployed Filebeat on the cluster, but I think it doesn't have a chance to work since in the /var/lib/...
- 109
0
votes
0
answers
6
views
Elasticsearch Lifecycle policy losing configuration
I have Kibana, Elasticsearch, and Filebeat running in an AKS cluster.
Filebeat is configured to capture logs from a few applications in my cluster, send to an index created each day, apply an ingest ...
0
votes
1
answer
66
views
Kubernetes filebeat config map for pod events
We have a pod that restarts randomly and we can't find the reason because Kubernetes only keeps event logs only for a short time. Even if we increase it, the logs will be lost when the pod is deleted.
...
- 1
0
votes
1
answer
940
views
Filebeat dial up error on localhost
So, I'm trying to configure Wazuh Server on a virtual machine (Ubuntu Desktop 22.04.1) and it needs the filebeat (without Elastic) to work correctly. I've installed both sucessfully and enabled them ...
- 53
0
votes
0
answers
323
views
How to have multiple instances of filebeat load balance Netflow input?
I have a very high volume Netflow input stream, and I was hoping that I could run multiple instances of Filebeat and load-balance the Netflow traffic over the Filebeat instances, and then write to a ...
- 211
0
votes
1
answer
148
views
Can logstash "pull" data?
I have two servers. Server A is running Elasticsearch and Logstash. Server B is running filebeat and is also the server which contains all the logs I'm trying to analyse.
Server A is behind a firewall,...
- 933
0
votes
1
answer
274
views
Filebeat on ECK with AWS Module Fails Due To Metadata Error
We are running an Elastic Stack with ECK in EKS (7.8). We noticed that our filebeat daemonset and the AWS module were not processing logs from S3 and our SQS queues backing up. Looking at the logs on ...
- 1
0
votes
1
answer
1k
views
filebeat log status 30 every sec
I'm learning to use ELK and have a debian PC that runs as a test client.
every 30 sec it logs a message :
021-01-18T08:29:59.656-0500#011INFO#011[monitoring]#011log/log.go:145#011Non-zero metrics in ...
- 115
0
votes
1
answer
191
views
How to use filebeat to find password in log
I have an Elasticsearch 7.1 and i have configure filebeat to collect all log.
I want to check if i have password in log.
So anyone have an idea how can i find all password in log using filebeat.
Thank ...
- 21
0
votes
1
answer
2k
views
Filebeat kafka input with SASL?
I'm trying to get filebeat to consume messages from kafka using the kafka input. I'm unable to authenticate with SASL for some reason and I'm not sure why that is. The documentation for both Kafka and ...
- 155
0
votes
1
answer
255
views
Integrating nginx logs and elasticsearch app-search
I'm trying to setup a self-managed docker appsearch instance, together with kibana and elasticsearch, queried by a uvicorn python app, proxied by a nginx webserver
My current issue is that the ...
- 121
0
votes
1
answer
654
views
Mapping fields from a beats log message in graylog
this is a slightly rephrased version of:
Whos is eating my fields? (or: how do I get more of the custom fields from my beats message into graylog)
i am using filebeat to collect logs from a bunch ...
- 176