Amazon EKS Kubernetes Version Rollback: How to Undo a Bad Cluster Upgrade
Amazon EKS Kubernetes Version Rollback: How to Undo a Bad Cluster Upgrade
On July 1, 2026, AWS shipped something EKS operators have wanted for years: the ability to actually undo a Kubernetes control plane upgrade. Until now, aws eks update-cluster-version was a one-way door โ if you upgraded from 1.32 to 1.33 and something broke (a deprecated API your controller depends on, a webhook that stopped working, an add-on that silently regressed), your only real options were to fix forward or rebuild the cluster. Most teams dealt with this by over-investing in bake periods, staggered upgrade waves, and month-long sign-off processes before touching production.
Version rollback changes that. You now get a 7-day window after any in-place upgrade to revert the control plane to the previous minor version, with AWS running automated compatibility checks before it lets you do it.
This post covers what actually gets rolled back, the CLI workflow, and the gotchas that aren't obvious from the announcement blog post.
What Rollback Actually Does (and Doesn't Do)
This is the part that trips people up. EKS rollback only touches the control plane. It is not a snapshot-and-restore of your whole cluster.
Gets rolled back:
- The Kubernetes API server version
- Control plane components and their configuration
- The platform version (reverts to the latest platform version for the previous K8s version)
- EKS Auto Mode worker nodes (automatically, before the control plane reverts)
Does NOT get rolled back:
- etcd data โ all your resources, state, and configs are preserved as-is
- Your workloads โ pods, deployments, services keep running
- EKS-managed add-ons โ versions stay wherever you left them, you manage separately
- Persistent volumes and data
- Self-managed nodes, hybrid nodes, and Managed Node Groups โ you roll these back yourself
That last point matters a lot. If your control plane goes from 1.33 back to 1.32 but your managed node group is still running 1.33 kubelets, you've now got nodes newer than the control plane, which violates the Kubernetes version skew policy. EKS will flag this before letting you proceed, but you still have to fix it manually.
Prerequisites
Amazon EKS won't let you attempt a rollback unless all of these hold:
| Requirement | Detail |
|---|---|
| 7-day window | Must initiate within 7 days of the upgrade completing |
| Was actually upgraded | Clusters created at their current version can't roll back โ only clusters that got there via in-place upgrade |
| One version only | N to N-1 only. Went 1.31 โ 1.32 โ 1.33? You can only roll back to 1.32, not 1.31 |
| Supported version | Target version must still be in standard or extended support |
| Extended support policy | Rolling back into extended support requires switching the cluster's upgrade policy to EXTENDED first |
| Cluster status | Must be ACTIVE, no other update in progress |
| Feature compatibility | If a feature you enabled isn't supported on the older version, rollback is blocked โ even with --force |
Step 1: Check Rollback Readiness Insights
After any upgrade, EKS automatically evaluates the cluster against a ROLLBACK_READINESS insight category and keeps those results available for the full 7-day window.
1aws eks list-insights \
2 --cluster-name my-cluster \
3 --region us-west-2 \
4 --filter '{"categories": ["ROLLBACK_READINESS"]}'
Drill into a specific finding:
1aws eks describe-insight \
2 --cluster-name my-cluster \
3 --region us-west-2 \
4 --id <insight-id>
These checks cover API usage compatibility (down to field-level changes), cluster health, kubelet/kube-proxy version skew, and add-on version compatibility. For Auto Mode clusters, EKS additionally checks NodePool disruption budgets, do-not-disrupt annotations, and PodDisruptionBudgets.
Insight statuses and what they mean for you:
| Status | Effect |
|---|---|
| PASSING | Rollback allowed |
| WARNING | Rollback allowed, advisory only |
| ERROR | Blocked unless you use --force |
| UNKNOWN | Blocked unless you use --force |
If insights look stale, force a refresh (EKS also auto-refreshes right before an actual rollback attempt):
1aws eks start-insights-refresh \
2 --cluster-name my-cluster \
3 --region us-west-2
Step 2: Handle Worker Nodes Before You Touch the Control Plane
- EKS Auto Mode: nothing to do โ EKS rolls back Auto Mode nodes automatically before reverting the control plane, respecting your disruption budgets.
- Managed Node Groups: you must downgrade them yourself, before rolling back the control plane:
1aws eks update-nodegroup-version \
2 --cluster-name my-cluster \
3 --nodegroup-name my-nodegroup \
4 --kubernetes-version 1.32 \
5 --region us-west-2
- Self-managed / hybrid nodes: also your responsibility โ swap AMIs or node configs to the older version yourself.
- Fargate: not supported for rollback. If Fargate pods are running the same K8s version as the control plane, they'll trigger a kubelet-skew ERROR insight. Delete and let them relaunch post-rollback, or force through it and accept the risk.
Step 3: Roll Back the Control Plane
Same API you already know, just pointed at the older version:
1aws eks update-cluster-version \
2 --name my-cluster \
3 --kubernetes-version 1.32 \
4 --region us-west-2
Example response:
1{
2 "update": {
3 "id": "e4091a28-ea14-48fd-a8c7-975aeb469e8a",
4 "status": "InProgress",
5 "type": "VersionRollback",
6 "params": [
7 {"type": "Version", "value": "1.32"},
8 {"type": "PlatformVersion", "value": "eks.16"}
9 ],
10 "createdAt": "2026-07-10T10:56:01.082000-04:00",
11 "errors": []
12 }
13}
Note the "type": "VersionRollback" โ EKS recognizes this as a distinct operation type, not just another version update.
If insight checks are showing ERROR but you've assessed the risk and want to proceed anyway:
1aws eks update-cluster-version \
2 --name my-cluster \
3 --kubernetes-version 1.32 \
4 --force \
5 --region us-west-2
--force only bypasses insight checks (ERROR/WARNING/UNKNOWN). It does not bypass the 7-day window, the "must have been upgraded" check, the sequential-rollback-only rule, or Auto Mode disruption controls. Those are hard stops regardless.
Step 4: Monitor It
1aws eks describe-update \
2 --name my-cluster \
3 --region us-west-2 \
4 --update-id e4091a28-ea14-48fd-a8c7-975aeb469e8a
Standard clusters transition InProgress โ Successful or InProgress โ Failed. Control plane rollback typically takes around 20 minutes. Auto Mode clusters stay ACTIVE while nodes roll back and only flip to UPDATING once the control plane rollback actually starts โ use describe-update rather than watching cluster status if you're on Auto Mode.
Best Practices
- Don't treat the insight check as a substitute for testing. Insights are evaluated at a point in time. If you create resources using new APIs after the check but before the rollback finishes, those changes aren't covered and can break post-rollback.
- Roll back node groups before the control plane, not after โ this order avoids running nodes newer than your control plane, even briefly.
- Downgrade incompatible add-ons first. EKS never auto-rolls-back add-on versions:
1aws eks update-addon \
2 --cluster-name my-cluster \
3 --addon-name vpc-cni \
4 --addon-version v1.12.0-eksbuild.2 \
5 --region us-west-2
- Watch extended-support billing. If you roll back from a standard-support version into one that's now in extended support, extended support charges start again immediately.
- Don't rely on CloudFormation stack rollback to fix your cluster. If a CFN stack update fails and rolls back to a template referencing an older K8s version, that does not trigger an EKS cluster rollback automatically โ you still have to call
update-cluster-versionexplicitly.
Common Mistakes to Avoid
- Assuming this is a full disaster-recovery mechanism. It isn't โ etcd data, workloads, and add-on versions are untouched. If your actual problem is corrupted cluster state, rollback won't help.
- Trying to skip a version. N-2 rollback is not supported. Upgraded 1.31 โ 1.32 โ 1.33 and want back to 1.31? You have to roll back to 1.32 first, then again to 1.31 if that path is still within its own separate 7-day window (it usually won't be, since the clock started at the 1.32โ1.33 upgrade).
- Forgetting Fargate. Teams that mix Fargate and EC2 node groups on one cluster get caught out by the kubelet skew insight on Fargate pods, since EKS can't roll those back for you.
- Using
--forcereflexively. It's tempting to slap--forceon every blocked rollback, but ERROR-status insights exist because something concrete is incompatible โ bypassing them can leave you in a worse state than the failed upgrade did. - Waiting past the 7-day window to decide. If you're mid-incident and unsure whether to roll back, don't sit on it โ the window is fixed from when the upgrade completed, not from when you started investigating.
Troubleshooting
Rollback request fails immediately with a feature-compatibility error, even with --force: Some EKS-level features enabled after the upgrade are hard-incompatible with the older control plane version. This check cannot be bypassed at all โ the fix is to disable the feature first, if that's viable, or accept you can't roll back.
describe-update shows Failed: Check errors in the response payload; the most common cause is a node group or add-on that was left running an incompatible version and blocked the control plane from settling.
Rollback succeeded but workloads are still broken: Remember the shared responsibility split โ EKS guarantees the control plane reverted correctly, but application-level compatibility with the older API surface is on you to verify.
FAQ
Does rolling back cost anything extra?
No. It uses the standard UpdateClusterVersion API at no additional charge, in all commercial regions where EKS is available. You only pay standard EKS/compute costs, plus extended support charges if the rollback lands you back in an extended-support version.
Can I roll back more than one minor version at once? No. Rollback is strictly N to N-1. Multi-hop rollbacks require multiple sequential operations, each within its own 7-day eligibility window.
What happens to my EKS-managed add-ons during rollback?
Nothing โ their versions are left exactly as they were. You need to check add-on compatibility with the target version yourself and downgrade with update-addon if needed.
Does this work with EKS Auto Mode? Yes, and it's actually smoother there โ EKS automatically rolls back Auto Mode worker nodes before reverting the control plane, honoring your NodePool disruption budgets. With Managed Node Groups or self-managed nodes, you handle node rollback yourself.
Can I still roll back after the cluster gets upgraded again? No. Once you upgrade a second time, the original rollback eligibility for the prior version is gone โ you'd only be able to roll back to the version immediately before the current one.
Key Takeaways
| Aspect | Detail |
|---|---|
| Rollback window | 7 days from upgrade completion |
| Scope | Control plane only โ one minor version at a time (N โ N-1) |
| Preserved automatically | etcd data, workloads, persistent volumes, add-on versions |
| Your responsibility | Managed/self-managed/hybrid node versions, add-on compatibility, app compatibility |
| Auto Mode | Nodes roll back automatically before control plane |
| Cost | No additional charge beyond standard EKS/compute (and extended support, if applicable) |
| API | Same update-cluster-version / describe-update calls you already use for upgrades |
This is a genuinely useful safety net, not a replacement for testing upgrades in staging first. Think of it as insurance for the failure mode you didn't catch in staging โ a deprecated API some third-party controller quietly depended on, or an add-on interaction nobody flagged in review.
Further Reading
- Provide Access to Other IAM Users and Roles to AWS EKS After Cluster Creation
- Docker vs Containerd: Understanding the Shift in Kubernetes 1.24+
- How to Update Your Kubernetes Application Container Registry Before the Community Registry Change
- AWS Docs: Rollback cluster to previous Kubernetes version
- AWS News Blog: Upgrade Amazon EKS clusters with confidence using Kubernetes version rollbacks