Kubernetes Clusters - Upstream (eg...kubeadm) Vs Downstream(eg..eks) | Overview.

Kubernetes Clusters - Upstream (eg...kubeadm) Vs Downstream (eg..eks) - Overview.

Scope:

• Intro,
• Upstream Kubernetes (eg kubeadm),
• Downstream Kubernetes (eg eks),
• What It Means,
• Characteristics,
• Examples,
• Comparison Table,
• When to Use Upstream Kubernetes (eg kubeadm, kind, kops),
• When to Use  Downstream Kubernetes (eg eks),
• Real-world examples of upstream and downstream Kubernetes clusters,
• Visualization of Code Flow,
• Summary.

Intro:

• The terms "upstream" and "downstream" in the context of Kubernetes clusters often refer to the direction of code flow, contributions, and customization of the cluster.
• Here's twtech Overview.

 Kubernetes Clusters – Upstream vs Downstream

 Upstream Kubernetes

 What It Means:

•         The official Kubernetes project maintained by the Cloud Native Computing Foundation (CNCF).
•        Source of truth: https://github.com/kubernetes/kubernetes
•         It’s the pure, open-source version without vendor modifications.

Characteristics:

•         Community-developed and maintained.
•         Vendor-neutral, portable across environments.
•         Releases, features, and bug fixes originate here.

 Example Use:

•         Building a custom Kubernetes distribution.
•         Running a lab or sandbox cluster using tools like kubeadm, kind, kops or minikube.

 Downstream Kubernetes

What It Means:

•         A modified or curated version of upstream Kubernetes provided by vendors.
•         Adds features, integrations, or enterprise support.

 Characteristics:

•         May include:
•    Custom dashboards
•    Security patches
•    Better cloud integrations
•    Proprietary extensions or operators
•         Follows upstream releases but may lag behind slightly.
•         May remove or restrict certain upstream features for stability or policy.

 Examples:

•         Amazon EKS (AWS's downstream distro of Kubernetes)
•         Red Hat OpenShift (based on upstream, but heavily customized)
•         Rancher Kubernetes Engine (RKE)

 Comparison Table

Feature	Upstream Kubernetes	Downstream Kubernetes (Vendor)
Source.	CNCF / Community	Based on upstream
Customization.	Raw Kubernetes	Vendor-specific
Support.	Community-driven	Enterprise support available
Release Cadence.	Rapid	May lag for stability/testing
Integrations.	Manual	Built-in (IAM, networking, etc)
Ideal For.	Custom builds, research, labs	Production, enterprise, managed

When to Use Each

twtech Uses Upstream Kubernetes if: Like kubeadm

•         twtech wants full control.
•         twtech is contributing to Kubernetes.
•         twtech is building a custom Kubernetes distribution.

twtech Uses Downstream Kubernetes if:  Like EKS

•         twtech wants a ready-to-use, production-ready, or managed solution.
•         twtech needs enterprise support, integrated logging, monitoring, or security.
•         twtech want to minimize operational overhead.

twtech Insights

• Here are real-world examples of upstream and downstream Kubernetes clusters, to help twtech clearly understand the distinction:

 Upstream Kubernetes Clusters (Open Source / Community-Based)

• These clusters run the vanilla, unmodified version of Kubernetes from the CNCF:

Example of Upstream Kubernetes Clusters	Description
kubeadm	Tool provided by the Kubernetes project to install and bootstrap upstream clusters. Used in self-hosted or lab environments.
kind	“Kubernetes IN Docker” – used for local development/testing using upstream Kubernetes.
minikube	Local, lightweight upstream Kubernetes cluster for development or learning.
k3s (by Rancher)	Lightweight upstream-compatible Kubernetes distro optimized for edge/IoT.
bare-metal clusters using upstream tarballs	Direct installs using official Kubernetes releases and binaries from GitHub.

 Downstream Kubernetes Clusters (Vendor-Modified / Managed)

• These are vendor-distributed or managed Kubernetes clusters that are based on (but often extend) upstream Kubernetes:

Example of downstream Kubernetes Vendor Cluster	Description
Amazon EKS	AWS-managed Kubernetes with VPC, IAM, Fargate, and CloudWatch integration.
Google Kubernetes Engine (GKE).	GCP-managed Kubernetes with tight integration into Google Cloud services.
Red Hat OpenShift.	Enterprise Kubernetes built on upstream but with added CI/CD, security, and operator lifecycle management.
VMware Tanzu Kubernetes Grid.	VMware’s enterprise Kubernetes distribution, tailored for vSphere and multi-cloud.
Rancher (RKE, RKE2).	Rancher-managed Kubernetes clusters that simplify and harden upstream Kubernetes.
Canonical Kubernetes (Charmed Kubernetes).	Downstream distro with additional automation and enterprise support from Canonical (Ubuntu).

 Visualization of Code Flow

twtech-Summary

Type	Examples	Use Case
Upstream.	kubeadm, kind,kops.	Test, development, research
Downstream.	EKS, OpenShift, Rancher.	Production, enterprise, cloud

Amazon EKS Node Types | Overview.

Amazon EKS Node Types - Overview.

Scope:

• Intro.
• Amazon EKS Node Types (Managed Node Groups) EC2-based,
• Amazon EKS Node Types Fargate (Serverless),
• Self-Managed Nodes (EC2-based, legacy option),
• Key Features,
• Use Cases,
• Summary Table.

Intro:

Amazon EKS supports three main node types for running twtech Kubernetes workloads:

 Amazon EKS Node Types

1 Managed Node Groups (EC2-based)

What it is:

•         AWS provisions and manages a group of EC2 instances on twtech behalf.
•         These are registered as worker nodes in twtech EKS cluster.

 Key Features:

•         Automatic updates (AMI patching).
•         Integrated with Auto Scaling Groups.
•         Launch templates and custom AMIs supported.
•         twtech manages the EC2 instance size, type, and capacity.

twtech Use Cases:

•         General workloads that require customization (e.g., GPU, high memory).
•         Scenarios where full control over the instance is needed.

2 Fargate (Serverless)

 What it is:

•         Serverless compute for containers.
•         twtech doesn’t provision or manage EC2 instances.
•         twtech defines Fargate profiles to run certain pods in Fargate.

 Key Features:

•         No servers to manage.
•         Per-pod billing (CPU & memory).
•         Isolation per pod using Firecracker microVMs.
•         Native IAM, logging, and VPC integration.

twtech Use Cases:

•         Lightweight, bursty, or infrequent workloads.
•         Dev/test environments.
•         twtech wants to avoid infrastructure management.

3  Self-Managed Nodes (EC2-based, legacy option)

 What it is:

•         twtech manually provision EC2 instances and join them to the EKS cluster.

 Key Features:

•         Full control of lifecycle and configuration.
•         twtech  Must handle patching, scaling, and health checks itself.
•         Requires knowledge of Kubernetes bootstrapping.

 Use Cases:

•         Legacy clusters.
•         Highly customized infrastructure needs.
•         Use with tools like Kops or custom automation.

 twtech Summary Table

Node Type	Managed by AWS	Serverless	Custom AMI       Support	Auto Scaling	Best for
Managed Node Group.	✅ Yes	❌ No	✅ Yes	✅ Yes	General-purpose workloads
Fargate.	✅ Yes	✅ Yes	❌ No	✅ Yes (via profile)	Serverless, simple workloads
Self-Managed.	❌ No	❌ No	✅ Yes	⚠️ Manual	Legacy or advanced customization

Elastic Kubernetes Service (EKS) | Overview & Hands-On.

  Elastic Kubernetes Service (EKS) - Overview & Hands-On.

Scope:

• Intro,
• The concept:  Amazon EKS,
• Key Features,
• EKS Architecture Overview,
• Amazon EKS Use cases,
• Pricing,
• When to Use EKS,
• EKS cluster vs ECS cluster vs Self-Managed Kubernetes cluster,
• When to Use Amazon EKS Cluster,
• When to Use  Amazon ECS Cluster,
• When to Use Self-Managed Kubernetes Cluster (Kubeadm).
• Project: Hands-on. 

Intro:

The concept:  Amazon EKS

• Amazon Elastic Kubernetes Service (EKS) is a managed Kubernetes service that makes it easy to deploy, manage, and scale containerized applications using Kubernetes on AWS infrastructure.
• Kubernetes is cloud-agnostic... it can be used with services of Multi-Cloud providers  like,  Azure, GCP and many more.

 Key Features

Feature	Description
Fully Managed Control Plane.	AWS manages the Kubernetes control plane (API servers, etcd), including scalability and availability.
Integration with AWS Services.	Native integration with IAM, VPC, CloudWatch, ELB, EBS, and more.
Supports EC2 and Fargate.	twtech can run workloads on either Amazon EC2 or AWS Fargate (serverless).
High Availability.	Control plane is automatically deployed across multiple Availability Zones.
Security.	IAM-based authentication, Kubernetes RBAC, and support for PrivateLink and encryption.

 EKS Architecture Overview

1. Control Plane (Managed by AWS)
• Kubernetes API server
• etcd database
• Control plane logs available via CloudWatch
2. Worker Nodes (Managed by twtech)
• EC2 Instances (self-managed or managed node groups)
• Fargate (serverless pods)
3. Networking
• Integrates with Amazon VPC
• Uses Amazon VPC CNI plugin for pod networking (pods get VPC IPs)

 How twtect Uses Amazon EKS

1. Create an EKS Cluster
• Via Console, CLI, CloudFormation, or Terraform
2. Provision Worker Nodes
• EC2 (via node groups) or Fargate profiles
3. Configure kubectl
• Connect to the EKS API using kubectl and the AWS CLI
4. Deploy Applications
• Use standard Kubernetes manifests (YAML)

 Pricing

• Control Plane: ~$0.10/hour per cluster
• Compute: Based on EC2 or Fargate pricing
• Other Costs: Networking, EBS, and logging (CloudWatch)

 When to Use EKS

• twtech should already be using Kubernetes and want a managed solution.
• twtech wants tight integration with AWS services.
• twtech needs multi-AZ, scalable, secure Kubernetes clusters.
• twtech wants to avoid managing the Kubernetes control plane, so aws can do the management.

EKS cluster vs ECS cluster vs Self-Managed Kubernetes cluster:

• Here's a clear comparison of Amazon EKS, Amazon ECS, and Self-Managed Kubernetes across key dimensions:

 EKS vs ECS vs Self-Managed Kubernetes

Feature / Criteria	Amazon EKS (Elastic Kubernetes Service)	Amazon ECS (Elastic Container Service)	Self-Managed Kubernetes
Control Plane Management.	Fully managed by AWS.	Fully managed by AWS.	twtech manages everything
Orchestration Engine.	Kubernetes.	AWS-native (not Kubernetes)	Kubernetes
Standards & Portability.	Open-source, portable across clouds.	AWS-specific.	Fully portable
Ease of Use.	Moderate (Kubernetes complexity exists).	Easier (simplified abstractions)	Harder (install, upgrade, maintain)
Cost for Control Plane.	~$0.10/hour per cluster.	Free.	Varies (depends on setup)
Compute Options.	EC2, Fargate.	EC2, Fargate.	Any (EC2, on-prem, other cloud)
Networking.	VPC CNI plugin (pods get VPC IPs).	ENIs for tasks.	Depends on configuration
Logging & Monitoring.	CloudWatch, Fluent Bit, Prometheus, etc.	CloudWatch.	twtch configures and manage
Auto Scaling.	K8s HPA, Cluster Autoscaler, Karpenter.	ECS Service Auto Scaling.	Requires manual setup
Deployment Options.	Declarative YAML (kubectl, Helm, etc.)	JSON/YAML or AWS. console/API.	Declarative YAML (kubectl)
CI/CD Integration.	Works well with GitOps (e.g., ArgoCD)	Works well with CodePipeline, CodeDeploy.	Full control, more setup
Security (IAM/RBAC).	IAM + Kubernetes RBAC.	IAM roles/tasks.	Manual RBAC & cert management
Use Case Fit.	Complex microservices, multi-cloud.	Simpler AWS-native workloads.	Custom infra, full control

 When to Use Amazon EKS Cluster

• twtech needs Kubernetes, but want AWS to manage the control plane.
• twtech is already using Kubernetes-native tooling (Helm, ArgoCD, etc.).
• twtech wants portability or hybrid/multi-cloud.

When to Use  Amazon ECS Cluster

• twtech wants the easiest way to run containers on AWS.
• twtech doesn’t need Kubernetes complexity.
• twtech workloads are AWS-centric and can be tightly coupled to AWS services.

 When to Use Self-Managed Kubernetes Cluster (Kubeadm)

• twtech needs full control over everything (e.g., for compliance).
• twtech is running on-prem, multi-cloud, or edge environments.
• twtech wants to experiment with low-level Kubernetes internals.

Project: Hands-on

• How twtech deploys and use Amazon EKS in its environment.

Step-1:

Search for aws services: Elastic Kubernetes Service

Step-2:

• Create EKS Cluster and assign a name: twtech-eks-cluster
• Configure cluster

• Create a IAM role to manage services in the cluster: twtech-eks-cluster-role
• Cluster IAM roleInfo
• Select the Cluster IAM role to allow the Kubernetes control plane to manage AWS resources on twtech behalf. 
• This cannot be changed after the cluster is created. 
• To create a new custom role.

• Select trusted entity: Trusted entit type.

• Select the type of aws service: EKS

• Auto Selected permissions

• Assign Name, review, and create

Step-3:

Go back to eks cluster configuration, refresh and select the role created: twtech-eks-cluster-role.

• Create: Node IAM role
• Nodes need an EC2 Instance IAM Role to launch and register with a cluster. 
• To create a new custom role, follow the instructions in the Amazon EKS

Step-4:

• Add permissions: EKSWorkerNodePolicy & EC2ReadOnlyAccess
• Name, review, and create

Assign node role name: twtech-node-role

Step-5:

Go back to eks cluster configuration, refresh and select the role created: twtech-node-role

From: creating

To: Active

Step-6:

twtech verifies all resources auto-provisioned with the eks cluster as defined in the launch template.

• Resources auto-provisioned with eks-cluster: security group

• When cluster has been fully created (it takes a couple of minutes to be fully provisioned),
• Next twtech needs to add the compute groups(target or node groups):twtech-eks-tg
• Select the cluster and click open to create node groups: twtech-node-group

Step-7:

• twtech Navigates to Compute-tab / Node group configuration to create the node group: twtech-node-group.

Configure node group

• A node group is a group of EC2 instances that supply compute capacity to twtech Amazon EKS cluster. 
• twtech can add multiple node groups to its cluster.
• twtech Assignsa name for the node group: twech-node-group

• Set compute and scaling configuration

• Node group scaling configuration

• Node group update configuration

• Specify networking

• Review and create

• From: creating

• Ec2 intanstances provisioned: 

Step-8:

twtech verifies the ASG created:

NB:

Fargate profiles(serverless) can also be added

• Step-9:
• How twtech may add add-ons to the eks cluster: twtech-eks-cluster.
• add-ons perform addinal rule to the eks cluster.

Select add-ons

• Review the add-ons from multiple categories, then select add-ons to enhance the functioning of the created cluster.

Community add-ons (5)

• When installing Amazon EKS Add-ons,  twtech carefully review the scope of support. 
• twtech is responsible for the open source or third party software (including community add-ons) that twtech may also install on the cluster.

Step-10:

• How twtech may choose to delete (Clean-up) its unwanted eks cluster: twtech-eks-cluster
• First, twtech deletes the node groups created and attached to the cluster: twtech-node-group
• Select the node group attached to the cluster and unwanted to: delete

• Type node group name to confirm deletion: twtech-node-group

NB:

• This action will delete the instances created with the node group.

• When the node group(twtech-node-group) is fully deleted, twtech prceeds to delete the entire cluster(twtech-eks-cluster) with all the resources created with it.

• Type the cluster name to confirm deletion: twtech-eks-cluster