EKS Multi-AZ Node Groups

Amazon EKS (Elastic Kubernetes Service) is a managed Kubernetes service that simplifies running Kubernetes on AWS. A node group is a collection of EC2 instances (worker nodes) that are registered with an EKS cluster and run your containerized applications. In a Multi-AZ node group, these worker nodes are distributed across two or more Availability Zones within the same AWS region. This distribution ensures that if one AZ fails, the application pods can be rescheduled onto nodes in the remaining AZs, providing high availability.

Kubernetes itself does not inherently understand AWS Availability Zones. Without explicit configuration, the Kubernetes scheduler might place all pods on nodes in a single AZ, creating a single point of failure. Multi-AZ node groups leverage AWS features (like EC2 Auto Scaling groups with multiple subnets) to spread nodes across AZs, and they use Kubernetes topology spread constraints or pod anti-affinity to ensure pods are also spread. For the SAA-C03, you must understand that this pattern is essential for meeting SLAs that require 99.99% uptime or for applications that must survive an AZ outage.

An EKS node group is backed by an EC2 Auto Scaling group (ASG). When you create a node group with multiple subnets (each in a different AZ), the ASG distributes EC2 instances across those subnets evenly by default. The EKS cluster then automatically labels each node with failure-domain.beta.kubernetes.io/zone (or topology.kubernetes.io/zone in newer versions) set to the AZ name (e.g., us-east-1a). The Kubernetes scheduler uses these labels to enforce pod placement policies.

Key components: - Subnets: You must specify at least two subnets in different AZs when creating the node group. Each subnet must have a route to the internet (if public) or a NAT gateway (if private). - Auto Scaling Group: The ASG manages the lifecycle of instances. It uses the specified launch template or launch configuration to create instances. - Node Group Scaling: You can set minimum, maximum, and desired size. The ASG attempts to keep the desired number of instances, distributing across AZs. - Kubernetes Labels: EKS automatically adds labels like topology.kubernetes.io/zone and failure-domain.beta.kubernetes.io/region. These are used by the scheduler for pod topology spread constraints.

To create a Multi-AZ node group via the AWS CLI, you can use the following command:

aws eks create-nodegroup \
    --cluster-name my-cluster \
    --nodegroup-name multi-az-ng \
    --scaling-config minSize=2,maxSize=10,desiredSize=3 \
    --subnets subnet-abc123 subnet-def456 subnet-ghi789 \
    --instance-types t3.medium \
    --node-role arn:aws:iam::123456789012:role/eks-node-role

Verification: After creation, check the node group status:

aws eks describe-nodegroup --cluster-name my-cluster --nodegroup-name multi-az-ng

Look for status: ACTIVE. To see the AZ distribution of nodes:

kubectl get nodes -o custom-columns=NAME:.metadata.name,ZONE:.metadata.labels.'failure-domain\.beta\.kubernetes\.io/zone'

To ensure pods are spread across AZs, you must use Kubernetes pod topology spread constraints or pod anti-affinity. For example:

apiVersion: apps/v1
kind: Deployment
spec:
  replicas: 6
  template:
    spec:
      topologySpreadConstraints:
      - maxSkew: 1
        topologyKey: topology.kubernetes.io/zone
        whenUnsatisfiable: DoNotSchedule

This ensures that pods are evenly distributed across zones, with a skew of at most 1 (i.e., no zone has more than one extra pod compared to another). Without such constraints, the scheduler might place all pods on nodes in the same AZ, even if nodes are spread.

For SAA-C03, you should know:

The Cluster Autoscaler (CA) can scale node groups up and down. When multiple node groups exist (e.g., one per AZ), the CA will consider each node group independently. However, with a single Multi-AZ node group, the CA triggers the ASG to add instances, and the ASG distributes them across AZs. This can lead to imbalance if the CA adds only one instance at a time (it goes to the AZ with the fewest instances). To mitigate, you can use node group per AZ or set the ASG's availability zone distribution to balanced best effort.

A large e-commerce company runs its customer-facing web application on EKS. The application requires 99.99% uptime and must survive an entire AZ failure. They deploy a Multi-AZ node group across three AZs (us-east-1a, us-east-1b, us-east-1c) with a minimum of 5 nodes per AZ (total 15 minimum). They use a managed node group with t3.large instances. The deployment uses pod topology spread constraints with maxSkew: 1 and whenUnsatisfiable: DoNotSchedule. They also use a Cluster Autoscaler to scale up during peak shopping seasons. In production, they observed that during a real AZ outage, the ASG automatically launched replacement nodes in the healthy AZs, and pods were rescheduled within 2 minutes. The key configuration was setting the ASG's capacity rebalancing to "balanced best effort" to avoid uneven distribution. A common mistake they encountered was forgetting to set the topologySpreadConstraints on their StatefulSets, which caused all database pods to land in one AZ, leading to data unavailability during an outage.

A fintech company runs a microservices architecture for payment processing. They have strict regulatory requirements for data residency within a single region but across multiple AZs. They use separate node groups per AZ (e.g., ng-az1, ng-az2, ng-az3) instead of a single Multi-AZ node group because they need fine-grained control over node sizing per AZ. Each node group has its own ASG and scaling policies. They use pod node affinity to pin pods to specific AZs. The downside is increased management overhead. They also use the Kubernetes topology.kubernetes.io/zone label to enforce that pods from a critical service are spread across all three AZs. During an audit, they demonstrated that no single AZ failure could cause more than 33% capacity loss, meeting the SLA.