I have Kibana, Elasticsearch, and Filebeat running in an AKS cluster.
Filebeat is configured to capture logs from a few applications in my cluster, send to an index created each day, apply an ingest pipeline, and also apply a lifecycle policy.
The ingest pipeline and lifecycle policy are created using terraform, and the index is generated by filebeat.
Part of my filebeat configuration:
output.elasticsearch:
host: '${NODE_NAME}'
hosts: '["${ELASTICSEARCH_HOST}"]'
username: '${ELASTICSEARCH_USERNAME}'
password: '${ELASTICSEARCH_PASSWORD}'
protocol: https
ssl.certificate_authorities: ["/usr/share/filebeat/certs/ca.crt"]
index: "${CLUSTER_ENV}-filebeat-%{[agent.version]}-%{+yyyy.MM.dd}"
pipeline: "${CLUSTER_ENV}-cluster-pipeline"
setup:
ilm: # index lifecycle management
enabled: true
policy_name: "${CLUSTER_ENV}-cluster-policy"
overwrite: true
template:
enabled: true
name: "${CLUSTER_ENV}-filebeat-%{[agent.version]}-%{+yyyy.MM.dd}"
pattern: "${CLUSTER_ENV}-filebeat-%{[agent.version]}-%{+yyyy.MM.dd}"
The thing is: it seems that each time on the creation of a new daily index, the lifecycle policy gets "recreated" losing any customizations done in by the terraform.
I already tried removing the overwrite: true from the ilm block, but I don't think anything changed.
By default, the documentation says it applies a filebeat lifecycle policy (which gets created), so by specifying a name, I was expecting to use the existing one I created. Then what seems to be happening is that filebeat is re-creating the policy each day, with default parameters.
Also saw some similar questions, which indicate that daily indexes are not encouraged. Could it be related to my problem?
