K8s, CSI-NFS, PVC и Statefulsets: «селектор для *v1.PersistentVolumeClaim не реализован»
Я использую https://github.com/kubernetes-csi/csi-driver-nfs , чтобы предоставить PVC NFS для моих наборов состояний. Но после нового автоматизированного развертывания (возможно, это происходит какое-то время, я не следил за кластером) журналы PVC сообщают следующее:
$ kubectl get pod,pvc,storageclass
NAME READY STATUS RESTARTS AGE
pod/kafka-0 0/1 Pending 0 17m
pod/kafka-1 0/1 Pending 0 17m
pod/kafka-2 0/1 Pending 0 17m
pod/nfs-server-595df7c955-gtsv9 1/1 Running 0 22m
pod/zookeeper-0 0/1 Pending 0 17m
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/kafka-kafka-0 Pending kafka 17m
persistentvolumeclaim/kafka-kafka-1 Pending kafka 17m
persistentvolumeclaim/kafka-kafka-2 Pending kafka 17m
persistentvolumeclaim/zookeeper-zookeeper-0 Pending zookeeper 17m
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
storageclass.storage.k8s.io/kafka nfs.csi.k8s.io Delete Immediate false 22m
storageclass.storage.k8s.io/zookeeper nfs.csi.k8s.io Delete Immediate false 22m
$ kubectl logs persistentvolumeclaim/zookeeper-zookeeper-0
error: cannot get the logs from *v1.PersistentVolumeClaim: selector for *v1.PersistentVolumeClaim not implemented
$ kubectl describe persistentvolumeclaim/zookeeper-zookeeper-0
Name: zookeeper-zookeeper-0
Namespace: meia
StorageClass:
Status: Pending
Volume:
Labels: app.kubernetes.io/component=zookeeper
app.kubernetes.io/instance=kafka
app.kubernetes.io/name=zookeeper
Annotations: <none>
Finalizers: [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
VolumeMode: Filesystem
Used By: zookeeper-0
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal FailedBinding 4s (x4 over 41s) persistentvolume-controller no persistent volumes available for this claim and no storage class is set
Любопытно, что эта же ошибка возникает и на локальном кластере Kind. Это происходит с несколькими версиями образов zookeeper/kafka и версиями 3.1 и 4.0 csi-nfs, это происходит либо в моем локальном кластере Kind, либо в моем чистом 5-узловом кластере kubernetes.
Это мой класс хранения:
$ cat storageclass-zookeeper.yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: zookeeper
labels:
name: zookeeper
provisioner: nfs.csi.k8s.io
parameters:
server: nfs-server.coucou.svc.cluster.local
share: /coucou-nfs/data/zookeeper
reclaimPolicy: Delete
volumeBindingMode: Immediate
mountOptions:
- hard
- nfsvers=4.1
И мой набор состояний:
$ cat statefulset-zookeeper.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: zookeeper
...
volumeMounts:
- name: scripts
mountPath: /scripts/setup.sh
subPath: setup.sh
- name: zookeeper
mountPath: /bitnami/zookeeper
volumes:
- name: scripts
configMap:
name: zookeeper-scripts
defaultMode: 0755
volumeClaimTemplates:
- metadata:
name: zookeeper
annotations:
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "zookeeper"
resources:
requests:
storage: 8Gi
Я не могу найти конкретную ошибку в Google. Итак, в чем проблема? Что-то с CSI-NFS? (проверено несколько версий). Проблемы с CRI/Containerd?
На моем локальном типе:
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short. Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.0", GitCommit:"4ce5a8954017644c5420bae81d72b09b735c21f0", GitTreeState:"clean", BuildDate:"2022-05-03T13:46:05Z", GoVersion:"go1.18.1", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.4
Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.4", GitCommit:"e6c093d87ea4cbb530a7b2ae91e54c0842d8308a", GitTreeState:"clean", BuildDate:"2022-03-06T21:32:53Z", GoVersion:"go1.17.7", Compiler:"gc", Platform:"linux/amd64"}
В моем кластере k8s:
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short. Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.0", GitCommit:"4ce5a8954017644c5420bae81d72b09b735c21f0", GitTreeState:"clean", BuildDate:"2022-05-03T13:46:05Z", GoVersion:"go1.18.1", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.4
Server Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.0", GitCommit:"4ce5a8954017644c5420bae81d72b09b735c21f0", GitTreeState:"clean", BuildDate:"2022-05-03T13:38:19Z", GoVersion:"go1.18.1", Compiler:"gc", Platform:"linux/amd64"}