0

In our GKE 1.27.12 cluster, we run a couple of stateful workloads using GCP Volumes, e.g. using this storage class:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: pd-ssd
provisioner: pd.csi.storage.gke.io
parameters:
  type: pd-ssd
reclaimPolicy: Delete
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer

However, the embedded Kubelet doesn't show any container_fs_* metrics (e.g. container_fs_writes_total) for pod mounting a volume. It only includes container_fs_* metrics for the root volume (aka. the container filesystem).

I compared with a 1.28.10 k3s cluster (kube-hetzner, using the hcloud-volumes storgage class), where the kubelet/cadvisor happily exports container_fs_* metrics for all mounted volumes.

Any idea how to make this work on GKE?

I tried running my own cadvisor without any luck, so I somehow suspect GCP to be the issue here...

Any help is much appreciated. :)

Improve this question
2
  • As per the question tags, it looks like you are using Prometheus, please integrate it with GKE and create custom metrics which may help to resolve your issue.
    – Kiran Kotturi
    Commented Jun 25 at 12:31
  • You are using a GKE version 1.27.12 which is not available as per the official release notes. Please use the stable version 1.27.11-gke.1062004 and let us know if the container_fs_* metrics are exported correctly.
    – Kiran Kotturi
    Commented Jun 25 at 12:31

0

Reset to default

You must log in to answer this question.