I'm trying to deploy a statefulset with 3 replicas on AWS EKS, with each replica having it's own persistent storage(on efs) dynamically created. The efs server address is stored in aws secrets manager.
But it's not working. I'm getting the following error:
Warning FailedScheduling 4m58s default-scheduler 0/1 nodes are available: pod has unbound immediate PersistentVolumeClaims. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling.
Any help is appreciated.
secrets-provider-class.yaml
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
name: aws-secrets-provider
spec:
provider: aws
parameters:
objects: |
- objectName: "efs-server-address"
objectType: secretsmanager
objectVersion: ""
efs-storageclass.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: efs-storageclass
provisioner: secrets-store.csi.k8s.io
parameters:
secretProviderClass: "aws-secrets-provider"
eks-secrets-manager-efs-sa.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: eks-secrets-manager-efs-sa
annotations:
eks.amazonaws.com/role-arn: arn:aws:iam::1234567890:policy/eks-secrets-manager-efs-policy
efs-pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: efs-pvc
spec:
accessModes:
- ReadWriteMany
storageClassName: efs-storageclass
resources:
requests:
storage: 1Gi
selector:
matchLabels:
fileSystemId: fs-0f012ab34567ab89c
statefulset-01.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: statefulset-01
labels:
app: app-sample-01
spec:
replicas: 3
selector:
matchLabels:
app: app-sample-01
template:
metadata:
labels:
app: app-sample-01
spec:
serviceAccountName: eks-secrets-manager-efs-sa
containers:
- name: container-sample-01
image: '123456789000.dkr.ecr.us-east-1.amazonaws.com/sample-repo:latest'
ports:
- containerPort: 8080
volumeMounts:
- name: efs-volume
mountPath: /data
volumeClaimTemplates:
- metadata:
name: efs-volume
spec:
accessModes:
- ReadWriteMany
storageClassName: efs-storageclass
resources:
requests:
storage: 1Gi