EKS Versioning: What You Need to Know and How to Stay Ahead of the Curve
Here at OpSourced, we specialize in managing EKS cluster upgrades for our clients, which means we’ve encountered and solved many common challenges users face—like navigating version deprecations, minimizing downtime, and avoiding unexpected costs.
If you’re wondering how to handle EKS versioning, you’re in the right place. This article will provide a clear overview of the EKS version life-cycle, share actionable insights from our years of hands-on experience with EKS upgrades, and help you avoid extended support costs.
Let’s dive in and make EKS upgrades simple.
Scheduled Maintenance: Perform the upgrade during a scheduled maintenance window to avoid disrupting live applications.
Testing: Test the upgrade in a non-production environment first to ensure compatibility and smooth transition.
Rollback Plan: Although upgrading an EKS cluster is a forward-only process, ensure you have a rollback or contingency plan if things don’t go as expected. For instance, use Auto Scaling Groups and EC2 instance snapshots for worker nodes, and create backups of critical data.
Understanding the AWS EKS Lifecycle
What does EKS stand for?
Amazon Elastic Kubernetes Service (EKS) is a managed Kubernetes service provided by AWS.
What is Amazon EKS?
Amazon Elastic Kubernetes Service (EKS) is a managed Kubernetes service that simplifies the process of running Kubernetes on AWS by managing the Kubernetes control plane. EKS removes the operational overhead of managing a Kubernetes infrastructure.
AWS EKS and Kubernetes Versioning:
Amazon Elastic Kubernetes Service (EKS) aligns with Kubernetes' release and support lifecycle, which follows an N-2 support policy:
Kubernetes releases a new minor version approximately every three months (e.g., 1.28, 1.29, 1.30).
Only the three most recent minor versions are supported at any given time.
When a new version (e.g., 1.31) is released, the oldest supported version (e.g., 1.28) is deprecated and eventually removed from EKS.
AWS EKS Lifecycle Timelines:
Each Kubernetes minor version is supported for approximately 14-15 months from its initial release.
AWS provides detailed timelines and notices for each version's deprecation. You can monitor these on the EKS Kubernetes Versions Documentation.
What This Means for EKS Users
Version Deprecation: AWS typically provides a 90-day notice before deprecating a Kubernetes version. After deprecation, clusters running that version will no longer receive updates or security patches.
Extended Support Costs: Running a deprecated version may incur extended support charges to maintain functionality.
Compatibility Risks: Outdated clusters might face issues with newer workloads, API changes, and integration with third-party tools.
Why Do Timely Upgrades Matter?
Security and Stability: New versions include critical security patches and performance improvements.
Access to New Features: Upgrades unlock new Kubernetes features and APIs, improving workload efficiency and developer productivity.
Avoid Disruption: Delaying upgrades can lead to rushed updates, increased downtime, or compatibility challenges when support is withdrawn.
Step-by-Step Guide to EKS Upgrades
Here's a step-by-step guide to EKS upgrades, complete with examples and pictures.
Step 1: Upgrade the EKS Control Plane
1. Open the AWS Management Console:
Go to the EKS Console.
2. Select the Cluster to Upgrade:
In the navigation pane, choose Clusters.
Select the cluster you want to upgrade from the list.
3. Start the Control Plane Upgrade:
On the Cluster Overview page, you’ll see the current cluster version of Kubernetes running on the control plane.
Click the Update button next to the Kubernetes version.
Choose the version you want to upgrade to from the dropdown.
Review the details and click Upgrade to start the process.
Notice that kubernetes version upgrades cannot be skipped. For example, if you're upgrading from cluster previous kubernetes version 1.28 to 1.30, you'll need to upgrade to cluster version 1.29 first before proceeding to 1.30.
4. Monitor the Upgrade:
The cluster control plane upgrade can take several minutes to complete. You can monitor the status from the Cluster Overview page.
Once the status changes to Active, the control plane upgrade is complete.
Under the hood: The EKS API Server Update Process
The Amazon Elastic Kubernetes Service (EKS) control plane is a managed service that handles Kubernetes API server operations. During an EKS kubernetes upgrade, the update process involves launching new API server nodes with the updated Kubernetes version. These nodes replace the existing ones seamlessly to ensure cluster continuity and minimize downtime.
How Does the API Server Update Work?
-
Launching New API Server Nodes:
Amazon EKS initiates the update by creating new API server nodes that run the updated Kubernetes version.
These new nodes temporarily coexist with the old ones until the transition is complete.
The update process is designed to ensure high availability by gradually switching traffic to the new API server nodes.
-
Decommissioning Old Nodes:
After the new API server nodes are fully functional and stable, EKS decommissions the older nodes.
This step minimizes risks and ensures that the cluster remains operational throughout the process.
-
IP Address Requirements:
During the update, Amazon EKS requires up to five available IP addresses from the subnets specified when the cluster was created.
These IP addresses are temporarily used by the new API server nodes to establish network connectivity and manage cluster traffic during the upgrade.
-
Transparent Transition:
The update process is transparent to users, with no action required for most workloads.
Kubernetes clients, such as kubectl, automatically connect to the new API servers without interruption.
Be aware that certain EKS cluster version upgrades are not permitted until the Node Group has been upgraded first.
Why Does This Happen?
EKS operates under Kubernetes versioning rules where the control plane and worker nodes (Node Groups) must be compatible:
Control Plane Compatibility: The control plane version must match or precede the Node Group version by at most one minor version.
Node Group Lag: Worker nodes running an older Kubernetes version might not support new API features in an upgraded control plane, potentially leading to cluster instability.
For example:
Supported: Control plane on 1.29 and Node Group on 1.28.
Unsupported: Control plane on 1.30 and Node Group still on 1.28.
Step 2: Upgrade Managed Node Groups
1. Go to the Node Group:
In the EKS Console, select your cluster.
Under the Compute section, click on Node groups.
Select the node group you wish to upgrade.
2. Start the Node Group Upgrade:
On the node group page, click the Update button next to the node group version.
Select the new Kubernetes version to upgrade the worker nodes.
Click Update to begin the upgrade process.
3. Monitor the Upgrade:
The upgrade process will roll out new worker nodes with the new cluster version, replacing the old nodes.
4. Validate Worker Nodes:
-
Once the upgrade is complete, ensure all worker cluster nodes are Ready by navigating to the Compute section and selecting Nodes.
Using EKS managed node groups simplifies node upgrades. The AWS Management Console allows you to roll out updates while maintaining high availability.
Step 3: Upgrade Kubernetes Add-ons
1. Navigate to the Add-ons:
In the EKS Console, select the Add-ons tab under the selected cluster.
Here you’ll see the current add-ons (such as CoreDNS, kube-proxy, and VPC CNI) installed on the cluster.
2. Upgrade Add-ons:
For each add-on, select it and click Update to upgrade to the latest version compatible with the upgraded Kubernetes version.
Review the update details and confirm the upgrade for each add-on.
3. Monitor the Add-on Upgrades:
The console will show the status of each add-on upgrade. Wait for the status to show as Active.
Step 4: Post-Upgrade Validation
1. Check Node Status:
In the AWS EKS Console, under the Compute section, Ensure that all worker nodes are healthy and Ready in the Compute section.
2. Check Application Status:
Validate that the applications running on the new kubernetes cluster version are functioning correctly by viewing the workloads and pods in the Kubernetes Dashboard (if enabled) or by using kubectl to check the status of the pods.
3. Monitor Logs and Metrics:
Use monitoring tools like Grafana to ensure cluster performance post-upgrade.
Additional Insights: Frequently Asked Questions About EKS Upgrades
How long does an EKS upgrade take?
EKS upgrades typically take several minutes to complete for the control plane and slightly longer for managed node groups, depending on the size of the cluster, while self managed node groups do upgrades one by one.
How to upgrade the EKS Cluster without downtime?
Carefully schedule upgrades during maintenance windows, use Auto Scaling Groups for redundancy, and ensure proper testing in staging environments.
How do I plan for an EKS upgrade?
Planning for an EKS upgrade involves several steps to ensure your Amazon EKS cluster and critical cluster components are updated efficiently and without disruption. Here’s a quick recap of the step-by-step approach:
Review the Kubernetes Release Calendar: Monitor the release calendar for upcoming Kubernetes cluster versions to identify when your current version may reach end-of-life. This will help you avoid incurring extended support costs.
Check Compatibility with Add-ons and Features: Ensure that useful site features like the AWS Load Balancer Controller, Amazon Elastic File System, and Cluster Autoscaler are compatible with the target Amazon EKS cluster version. Reviewing relevant upgrade documentation is critical.
Allocate Resources and Schedule Downtime: Plan the upgrade during a maintenance window to ensure useful site features like networking, API server nodes, and workload node upgrades function properly without impacting users.
Communicate with Stakeholders: Notify internal teams, application owners, and end-users about the planned upgrade timeline and any potential impact on cluster operations. If you rely on the Amazon EKS cluster for critical workloads or user-facing applications, ensure all stakeholders are informed about the upgrade.
Test in a Staging Environment: Always validate upgrades in a staging cluster before applying them to production. Simulate workloads and evaluate metrics like latency and performance using tools that can collect anonymous statistics and perform analytics.
Prepare Rollback and Backup Plans: Use approved third parties or native tools like Amazon Elastic Kubernetes Service snapshots to back up critical data. This ensures you can save preferences or restore the system if issues arise.
How do I ensure my EKS cluster functions properly during upgrades?
Plan Redundancy: Leverage multiple versions of worker nodes and utilize Auto Scaling Groups to minimize risks during node upgrades.
Validate Networking: Ensure that the Amazon EKS cluster subnets have enough available IPs for new API server nodes.
Monitor Metrics: Use tools like AWS Console Footer logs or anonymous statistics dashboards to verify that all components function properly.
Check Application Performance: Confirm that your Kubernetes cluster workloads, such as applications using the Cluster Autoscaler, are stable.
How do I upgrade my EKS cluster with Terraform?
Use the Terraform AWS provider module to define and update the Kubernetes version for your EKS cluster and node groups. Apply changes to perform upgrades.
In Closing
At OpSourced, we specialize in simplifying EKS upgrades, ensuring minimal downtime and maximum efficiency.
By following these best practices, you can stay ahead of version deprecations and avoid extended support costs.
Contact us today to learn how we can streamline your EKS upgrade process.