728x90
반응형
EKS 1.22 버전을 사용하다 1.23 버전으로 업그레이드 하면 기본 EKS에서 제공하는 StorageClass(gp2)를 사용하지 못하고 ebs-csi-driver를 설치하여 사용할 수 있다. (참고 : https://docs.aws.amazon.com/ko_kr/eks/latest/userguide/ebs-csi-migration-faq.html )
ebs-csi-driver는 serviceaccount를 통해 아래 정책을 사용하여 ebs에 접근하기 때문에 다음과 같은 정책을 생성해야 한다.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:CreateSnapshot",
"ec2:AttachVolume",
"ec2:DetachVolume",
"ec2:ModifyVolume",
"ec2:DescribeAvailabilityZones",
"ec2:DescribeInstances",
"ec2:DescribeSnapshots",
"ec2:DescribeTags",
"ec2:DescribeVolumes",
"ec2:DescribeVolumesModifications"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"ec2:CreateTags"
],
"Resource": [
"arn:aws:ec2:*:*:volume/*",
"arn:aws:ec2:*:*:snapshot/*"
],
"Condition": {
"StringEquals": {
"ec2:CreateAction": [
"CreateVolume",
"CreateSnapshot"
]
}
}
},
{
"Effect": "Allow",
"Action": [
"ec2:DeleteTags"
],
"Resource": [
"arn:aws:ec2:*:*:volume/*",
"arn:aws:ec2:*:*:snapshot/*"
]
},
{
"Effect": "Allow",
"Action": [
"ec2:CreateVolume"
],
"Resource": "*",
"Condition": {
"StringLike": {
"aws:RequestTag/ebs.csi.aws.com/cluster": "true"
}
}
},
{
"Effect": "Allow",
"Action": [
"ec2:CreateVolume"
],
"Resource": "*",
"Condition": {
"StringLike": {
"aws:RequestTag/CSIVolumeName": "*"
}
}
},
{
"Effect": "Allow",
"Action": [
"ec2:CreateVolume"
],
"Resource": "*",
"Condition": {
"StringLike": {
"aws:RequestTag/kubernetes.io/cluster/*": "owned"
}
}
},
{
"Effect": "Allow",
"Action": [
"ec2:DeleteVolume"
],
"Resource": "*",
"Condition": {
"StringLike": {
"ec2:ResourceTag/ebs.csi.aws.com/cluster": "true"
}
}
},
{
"Effect": "Allow",
"Action": [
"ec2:DeleteVolume"
],
"Resource": "*",
"Condition": {
"StringLike": {
"ec2:ResourceTag/CSIVolumeName": "*"
}
}
},
{
"Effect": "Allow",
"Action": [
"ec2:DeleteVolume"
],
"Resource": "*",
"Condition": {
"StringLike": {
"ec2:ResourceTag/kubernetes.io/cluster/*": "owned"
}
}
},
{
"Effect": "Allow",
"Action": [
"ec2:DeleteSnapshot"
],
"Resource": "*",
"Condition": {
"StringLike": {
"ec2:ResourceTag/CSIVolumeSnapshotName": "*"
}
}
},
{
"Effect": "Allow",
"Action": [
"ec2:DeleteSnapshot"
],
"Resource": "*",
"Condition": {
"StringLike": {
"ec2:ResourceTag/ebs.csi.aws.com/cluster": "true"
}
}
}
]
}
KMS를 사용하여 암호화하는 경우 추가 정책이 필요하다
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"kms:CreateGrant",
"kms:ListGrants",
"kms:RevokeGrant"
],
"Resource": ["{custom-key-id}"],
"Condition": {
"Bool": {
"kms:GrantIsForAWSResource": "true"
}
}
},
{
"Effect": "Allow",
"Action": [
"kms:Encrypt",
"kms:Decrypt",
"kms:ReEncrypt*",
"kms:GenerateDataKey*",
"kms:DescribeKey"
],
"Resource": ["{custom-key-id}"]
}
]
}
EKS클러스터 구성할 때 자격 증명 공급자를 생성하지 않았으면 생성해야 한다.
Iam role을 생성하여 위에 정책을 추가하고 아래 신뢰 관계를 추가한다.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::{account_id}:oidc-provider/oidc.eks.{region code}.amazonaws.com/id/{oidc}"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"oidc.eks.{region code}.amazonaws.com/id/{oidc}:sub": "system:serviceaccount:kube-system:ebs-csi-controller-sa",
"oidc.eks.{region code}.amazonaws.com/id/{oidc}:aud": "sts.amazonaws.com"
}
}
}
]
}
Helm을 사용하여 설치한다. 설치할 때 image.repository를 지정해야 하는데 리전마다 repository 주소가 다르기 때문에 아래 AWS RegionRegistry 참고하여 설정한다.
$ helm repo add aws-ebs-csi-driver https://kubernetes-sigs.github.io/aws-ebs-csi-driver
$ helm upgrade -install aws-ebs-csi-driver aws-ebs-csi-driver/aws-ebs-csi-driver \
--namespace kube-system \
--set controller.k8sTagClusterId={eks-cluster-name} ## ebs Name tag 생성
--set image.repository={region registry}/eks/aws-ebs-csi-driver \
--set controller.serviceAccount.create=true \
--set controller.serviceAccount.name=ebs-csi-controller-sa \
--set controller.serviceAccount.annotations."eks\.amazonaws\.com/role-arn"="{iam_role_arn}"[ AWS RegionRegistry ]
| af-south-1 | 877085696533.dkr.ecr.af-south-1.amazonaws.com |
| ap-east-1 | 800184023465.dkr.ecr.ap-east-1.amazonaws.com |
| ap-northeast-1 | 602401143452.dkr.ecr.ap-northeast-1.amazonaws.com |
| ap-northeast-2 | 602401143452.dkr.ecr.ap-northeast-2.amazonaws.com |
| ap-northeast-3 | 602401143452.dkr.ecr.ap-northeast-3.amazonaws.com |
| ap-south-1 | 602401143452.dkr.ecr.ap-south-1.amazonaws.com |
| ap-south-2 | 900889452093.dkr.ecr.ap-south-2.amazonaws.com |
| ap-southeast-1 | 602401143452.dkr.ecr.ap-southeast-1.amazonaws.com |
| ap-southeast-2 | 602401143452.dkr.ecr.ap-southeast-2.amazonaws.com |
| ap-southeast-3 | 296578399912.dkr.ecr.ap-southeast-3.amazonaws.com |
| ap-southeast-4 | 491585149902.dkr.ecr.ap-southeast-4.amazonaws.com |
| ca-central-1 | 602401143452.dkr.ecr.ca-central-1.amazonaws.com |
| cn-north-1 | 918309763551.dkr.ecr.cn-north-1.amazonaws.com.cn |
| cn-northwest-1 | 961992271922.dkr.ecr.cn-northwest-1.amazonaws.com.cn |
| eu-central-1 | 602401143452.dkr.ecr.eu-central-1.amazonaws.com |
| eu-central-2 | 900612956339.dkr.ecr.eu-central-2.amazonaws.com |
| eu-north-1 | 602401143452.dkr.ecr.eu-north-1.amazonaws.com |
| eu-south-1 | 590381155156.dkr.ecr.eu-south-1.amazonaws.com |
| eu-south-2 | 455263428931.dkr.ecr.eu-south-2.amazonaws.com |
| eu-west-1 | 602401143452.dkr.ecr.eu-west-1.amazonaws.com |
| eu-west-2 | 602401143452.dkr.ecr.eu-west-2.amazonaws.com |
| eu-west-3 | 602401143452.dkr.ecr.eu-west-3.amazonaws.com |
| me-south-1 | 558608220178.dkr.ecr.me-south-1.amazonaws.com |
| me-central-1 | 759879836304.dkr.ecr.me-central-1.amazonaws.com |
| sa-east-1 | 602401143452.dkr.ecr.sa-east-1.amazonaws.com |
| us-east-1 | 602401143452.dkr.ecr.us-east-1.amazonaws.com |
| us-east-2 | 602401143452.dkr.ecr.us-east-2.amazonaws.com |
| us-gov-east-1 | 151742754352.dkr.ecr.us-gov-east-1.amazonaws.com |
| us-gov-west-1 | 013241004608.dkr.ecr.us-gov-west-1.amazonaws.com |
| us-west-1 | 602401143452.dkr.ecr.us-west-1.amazonaws.com |
| us-west-2 | 602401143452.dkr.ecr.us-west-2.amazonaws.com |
ebs-csi-driver파드가 잘 생성되었는지 확인한다.
$ kubectl get po -n kube-system | grep ebs-csi
ebs-csi-controller-7cbfb8577f-ljtqv 5/5 Running 0 26h
ebs-csi-node-ghgr4 3/3 Running 0 26h
storageclass를 생성한다.
$ kubectl apply -f - <<EOF
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: ebs-sc
provisioner: ebs.csi.aws.com
volumeBindingMode: WaitForFirstConsumer
parameters:
type: gp3
EOF
임의의 Pod를 생성하여 storageclass를 통한 볼륨 생성 테스트를 해본다.
$ kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
name: app
spec:
containers:
- name: app
image: centos
command: ["/bin/sh"]
args: ["-c", "while true; do echo $(date -u) >> /data/out.txt; sleep 5; done"]
volumeMounts:
- name: persistent-storage
mountPath: /data
volumes:
- name: persistent-storage
persistentVolumeClaim:
claimName: ebs-claim
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: ebs-claim
spec:
accessModes:
- ReadWriteOnce
storageClassName: ebs-sc
resources:
requests:
storage: 30Gi
EOF
다음과 같이 ebs-csi-controller 파드의 로그를 확인할 수 있다.
$ kubectl logs -n kube-system deployments/ebs-csi-controller -f
Waiting for volume state" volumeID="vol-0b3d5bb661f54ccc0" actual="attaching" desired="attached
aws console에서 정상적으로 프로비저닝 된 걸 확인할 수 있다.
위 방법 외에도 AWS Console상에서도 추가할 수 있다.
위에서 생성한 Role을 추가하고 생성하면 된다.
반응형
'Orchestration > Kubernetes' 카테고리의 다른 글
| AWS EKS 노드그룹 자동 시작, 중지하기 (0) | 2023.10.17 |
|---|---|
| EKS add-on(vpc-cni) 업그레이드 이슈 (0) | 2023.05.24 |
| EKS Pod 전용 Security group (SecurityGroupPolicy) (0) | 2023.05.18 |
| Kubernetes Pod의 Graceful 한 종료 (0) | 2023.04.15 |
| AWS EKS Security Group 최소 규칙 (0) | 2023.04.11 |
댓글