Amazon EKS Data Volumes | Overview.

Amazon EKS Data Volumes - Overview.

Sope:

• Intro,
• Types of Volumes in EKS,
• Amazon EBS (Elastic Block Store),
• Amazon EFS (Elastic File System),
• Ephemeral Volumes (emptyDir, hostPath, etc.),
• FSx for Lustre,
• Insights: Persistent Volume (PV) and Persistent Volume Claim (PVC),
• Dynamic Volume Provisioning in EKS,
• Sample Yaml file to provision a StorageClass for EBS,
• Best Practices.

Intro:

• In Amazon Elastic Kubernetes Service (EKS), data volumes refer to storage volumes used by pods to persist and share data.
• Volumes in Kubernetes, including those on EKS, allow containers to store and retrieve data across restarts, re-creations, or multiple container instances.

 Types of Volumes in EKS

• EKS supports multiple volume types through Kubernetes:

1. Amazon EBS (Elastic Block Store)

• Usage: Most common for persistent data storage.
• Characteristics:
• Block-level storage.
• Attached to a single EC2 node (i.e., a pod cannot move across nodes unless EBS is reattached).
• Supports ReadWriteOnce (RWO) access mode.
• Use Cases: Databases, stateful apps (e.g., MongoDB, PostgreSQL).

2. Amazon EFS (Elastic File System)

• Usage: Shared file storage across multiple pods and nodes.
• Characteristics:
• POSIX-compliant NFS file system.
• Multiple pods across AZs can read/write to the same volume.
• Supports ReadWriteMany (RWX).
• Use Cases: Shared config files, web content, media libraries.

3. Ephemeral Volumes (emptyDir, hostPath, etc.)

• Usage: Temporary storage tied to pod lifecycle.
• Characteristics:
• Data lost when pod is deleted.
• Useful for caches, scratch data.
• Examples:
• emptyDir: Created when pod starts.
• hostPath: Maps a host node directory into the pod.

4. FSx for Lustre

• Usage: High-performance shared storage.
• Use Case: HPC, ML workloads requiring fast, parallel I/O.

 Insights: Persistent Volume (PV) and Persistent Volume Claim (PVC)

To use volumes in Kubernetes:

1. PersistentVolume (PV): Abstraction of actual storage.
2. PersistentVolumeClaim (PVC): Request for storage by a pod.

Kubernetes schedules PVCs to match available PVs or triggers dynamic provisioning (e.g., automatically creating an EBS volume).

 Dynamic Volume Provisioning in EKS

EKS supports dynamic provisioning using StorageClasses, which define:

• The type of storage (EBS, EFS).
• Parameters (volume type, size, IOPS, etc.).
• Provisioner (e.g., ebs.csi.aws.com for EBS CSI driver).

Sample Yaml file to provision a StorageClass for EBS:

# yaml

apiVersion: storage.k8s.io/v1

kind: StorageClass

metadata:

  name: twtech-ebs-sc

provisioner: ebs.csi.aws.com

parameters:

  type: gp3

  fsType: ext4

 twtech-Best Practices

• Use EBS for stateful single-node workloads, like databases.
• Use EFS for shared, read/write access across pods.
• Use StorageClasses for dynamic provisioning.
• Use CSI drivers (EBS CSI, EFS CSI) for better management and flexibility.
• Monitor usage and automate volume resizing when needed.

 CSI (Container Storage Interface) Drivers in EKS

EKS supports CSI (Container Storage Interface) drivers:

• Amazon EBS CSI driver: For managing EBS volumes.
• Amazon EFS CSI driver: For EFS with Fargate integration.
• Amazon FXs CSI driver: For Luster & NetApp ONTAP
• Installed via EKS Add-ons or Helm charts.