PV, PVC의 동적 프로비저닝을 위한 NFS CSI 드라이버 구성

Prerequisite

1. NFS 서버/클라이언트 설정

nfs-provisioner 구성

helm repo add nfs-subdir-external-provisioner https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner/

helm repo update

kubectl create ns nfs-provisioner

helm install nfs-subdir-external-provisioner nfs-subdir-external-provisioner/nfs-subdir-external-provisioner \
    --namespace nfs-provisioner \
    --set nfs.server=woobin-vm1 \
    --set nfs.path=/mnt/repo

kubectl get storageclass

NAME         PROVISIONER                                     RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
nfs-client   cluster.local/nfs-subdir-external-provisioner   Delete          Immediate           true                   42m

Storage Class 테스트

# nfs-pvc.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: nfs
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: "nfs-client"
  resources:
    requests:
      storage: 4Gi

kubectl apply -f nfs-pvc.yaml
kubectl get pv,pvc

NAME                                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM         STORAGECLASS   REASON   AGE
persistentvolume/pvc-9d5d042b-bb50-4087-98c0-d8fdb7f0d70e   4Gi        RWX            Delete           Bound    default/nfs   nfs-client              62s

NAME                        STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
persistentvolumeclaim/nfs   Bound    pvc-9d5d042b-bb50-4087-98c0-d8fdb7f0d70e   4Gi        RWX            nfs-client     63s

위와 같이 PVC만 생성하여도 PV가 정상적으로 생성

ls -al /mnt/repo/

total 0
drwxrwxrwx  3 root root 66 May  3 09:12 .
drwxr-xr-x. 3 root root 18 May  2 15:21 ..
drwxrwxrwx  2 root root  6 May  3 09:12 default-nfs-pvc-9d5d042b-bb50-4087-98c0-d8fdb7f0d70e

위와 같이 NFS 서버에서도 확인

kubectl delete pvc nfs

테스트 후 pvc 삭제

참고) pvc 생성 시 access denied 이슈

pvc를 생성하는 Pod가 계속 Init stuck 상태 발생. 아래와 같은 describe 확인.

Output: mount.nfs: access denied by server while mounting woobin-vm1:/mnt/repo/monitoring-prometheus-kube-prometheus-stack-prometheus-db-prometheus-kube-prometheus-stack-prometheus-0-pvc-ba3f5202-a49f-42fe-83a5-026dbdac7046

/etc/exports에 nfs server도 설정되어 있어야 함. 안되어 있으면 해당 노드 pvc 생성 시 위와 같은 에러 발생

/mnt/repo woobin-vm1(rw,sync,no_root_squash,no_subtree_check)
/mnt/repo woobin-vm2(rw,sync,no_root_squash,no_subtree_check)
/mnt/repo woobin-vm3(rw,sync,no_root_squash,no_subtree_check)

Reference

1. https://kubernetes.io/ko/docs/concepts/storage/persistent-volumes/
2. https://kubernetes.io/ko/docs/concepts/storage/storage-classes/#nfs
3. https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner/blob/master/charts/nfs-subdir-external-provisioner/README.md