0

My stack is on AWS Fargate. Application, grafana, loki on the same ECS cluster. My application generates logs,

Log Stream sample below:

{"host":"ip-00-3001-1a08-test.us-west-1.compute.internal","short_message":"Sent message: {\"test\":{\"test\":\"31901300\",\"}}","full_message":"Sent message: {\"test\":{\"test\":\"31901300\"}}","timestamp":1.707740449767E9,"level":6,"facility":"logstash-abctest","simpleLogger":"test-abc","logSequence":9696969696,"activity":"00000000222255544","logLevel":"INFO","logger":"test","senderType":"test","terminalID":"test300"}

Find my logConfiguration in ECS Task_definitation:

"logConfiguration": {
                "logDriver": "awsfirelens",
                "options": {
                    "LabelKeys": "container_name,ecs_task_definition,source,ecs_cluster",
                    "Labels": "{job=\"firelens\"}",
                    "LineFormat": "json",
                    "Name": "grafana-loki",
                    "RemoveKeys": "container_id,ecs_task_arn",
                    "Url": "http://loki.endpoint:3100/loki/api/v1/push"
                }

There are some fields in my Log stream such as host, short_message, full_message, timestamp, level, facility, logSequence, activity, logLevel, logger, sendType, terminalID etc.

My goal is to parse these fields as Labels in Loki data source. Suggest necessary configuration to make in AWS ECS Task Definitation to accomplish it.

FYI, Currently, I am only able see meta data of cluster (container_name, ecs_cluster, ecs_task_definition, job, source) as Lebels.

Thanks in advance.

I tried to add fields of log stream in logConfiguration of the task definitation like this:

"LabelKeys": "senderType,activity,full_message,short_message,container_name,ecs_task_definition,source,ecs_cluster",
"Labels": "{job=\"firelens\",senderType=\"${senderType}\",activity=\"${activity}\"}",

But that didn't work.

0

You must log in to answer this question.

Browse other questions tagged .