ASG Termination Policies

An Auto Scaling Group (ASG) termination policy determines which EC2 instance is terminated when the group scales in (reduces capacity) or when an instance is marked unhealthy. The policy is applied during scale-in events triggered by scaling policies (e.g., target tracking, step scaling) or manual adjustments. The default termination policy is designed to distribute instances evenly across Availability Zones (AZs) and to terminate instances that are least likely to disrupt applications.

Termination policies directly impact application availability, cost, and performance. A poorly chosen policy can lead to:

The SAA-C03 exam tests your ability to select the correct termination policy for a given scenario, often combined with other ASG features like lifecycle hooks, health checks, and mixed instances policies.

When a scale-in event occurs, the ASG follows these steps: 1. Determine which AZ(s) to terminate from: The ASG first checks the AZ rebalancing logic. If the group is not balanced (i.e., one AZ has significantly more instances than others), it will terminate instances from the AZ with the most instances to restore balance. If all AZs are balanced, it proceeds to the next step. 2. Apply the termination policy: The ASG evaluates the configured termination policy (or the default if none is specified) to select an instance to terminate. The policy is applied in a specific order, as described below. 3. Consider lifecycle hooks: If a lifecycle hook for termination (autoscaling:EC2_INSTANCE_TERMINATING) is configured, the ASG waits for the hook to complete (or timeout) before proceeding. The default timeout is 3600 seconds (1 hour), configurable up to 48 hours. 4. Execute termination: The ASG calls the EC2 API to terminate the selected instance. The instance enters the shutting-down and then terminated state.

The default termination policy is designed to:

The exact algorithm: 1. Select the AZ with the most instances (if the group spans multiple AZs). If multiple AZs have the same count, the ASG randomly picks one among them. 2. Within that AZ, select instances with the oldest launch configuration/template (i.e., the earliest launch time). If multiple instances have the same launch time, select the one closest to the next billing hour. 3. If still tied, select randomly.

You can configure one of the following policies directly on the ASG:

The ASG evaluates policies in a specific order. If you specify multiple policies (up to 20), they are evaluated in sequence. The first policy that can select a unique instance wins. If no unique instance is selected, the next policy is applied. This allows you to create complex termination logic.

For example, if you set policies to: OldestLaunchTemplate, ClosestToNextInstanceHour, the ASG will first try to find an instance with the oldest launch template. If multiple instances have the same launch template age (e.g., all were launched with the same template), it will then apply the next policy to break the tie.

ASG uses health checks to determine if an instance is healthy. If an instance fails a health check (e.g., EC2 status check, ELB health check), the ASG marks it as unhealthy and will terminate and replace it. The termination policy is still applied when the ASG decides which unhealthy instance to terminate, but typically the unhealthy instance is the only candidate. However, if multiple instances are unhealthy, the policy is used to choose among them.

Lifecycle hooks can pause the termination process. When a lifecycle hook is in the autoscaling:EC2_INSTANCE_TERMINATING transition, the instance remains in the Terminating:Wait state until the hook completes or times out. This allows you to perform custom actions (e.g., backup logs, drain connections) before termination. The default timeout is 3600 seconds, but you can adjust it up to 48 hours.

You can put an instance in standby state. Instances in standby are not considered for termination during scale-in events. They also do not count as part of the desired capacity, so you can manually manage them without triggering scaling actions.

You can suspend the Terminate process on the ASG, which prevents any terminations. This is useful during maintenance windows or when you want to manually control scaling.

Consider an ASG with: - 6 instances across 3 AZs: AZ-a has 3, AZ-b has 2, AZ-c has 1. - All instances use the same launch template. - A scale-in event reduces desired capacity by 1.

The ASG will: 1. Select AZ-a (the AZ with the most instances). 2. Among the 3 instances in AZ-a, select the one with the oldest launch template (all same), then the one closest to the next billing hour. 3. Terminate that instance.

If the group was balanced (2 each), the ASG would randomly pick an AZ (or use the next policy if configured) and then apply the same logic.

You can set termination policies via the AWS Management Console, CLI, or SDK. For example, using the AWS CLI:

aws autoscaling update-auto-scaling-group \
    --auto-scaling-group-name my-asg \
    --termination-policies "OldestLaunchTemplate" "ClosestToNextInstanceHour"

You can also specify policies during creation:

aws autoscaling create-auto-scaling-group \
    --auto-scaling-group-name my-asg \
    --launch-template LaunchTemplateName=my-template \
    --min-size 1 --max-size 5 --desired-capacity 2 \
    --vpc-zone-identifier subnet-abc,subnet-def \
    --termination-policies "Default"

To see which termination policy is in effect:

aws autoscaling describe-auto-scaling-groups --auto-scaling-group-names my-asg

Look for the TerminationPolicies field in the output.

When using a mixed instances policy (e.g., combining On-Demand and Spot Instances), the termination policy applies uniformly across all instances. However, the ASG may prioritize terminating Spot Instances over On-Demand if the Spot allocation strategy requires it (e.g., if using capacity-optimized and Spot capacity is needed elsewhere). But the termination policy itself does not differentiate by instance type.

In a large e-commerce platform, ASG termination policies are critical for maintaining application stability during deployments. For example, when a new version of the application is released via a rolling update, the ASG might use the OldestLaunchTemplate policy to gradually replace instances running the old version. This ensures that only instances with the old template are terminated first, minimizing the number of instances running outdated code. However, if the new version has a bug, the team might want to quickly roll back by terminating the newest instances. In that case, they would temporarily switch the termination policy to NewestInstance.

Another scenario involves cost optimization. A company runs a mix of On-Demand and Spot Instances in the same ASG. They want to ensure that Spot Instances are terminated before On-Demand when scaling in, to save costs. However, the termination policy itself does not distinguish between instance types. To achieve this, they use a lifecycle hook that checks the instance's lifecycle (Spot vs. On-Demand) and, if it's On-Demand, puts it back into service or delays termination. Alternatively, they could use two separate ASGs—one for On-Demand and one for Spot—each with its own termination policy.

A common misconfiguration is setting the termination policy to ClosestToNextInstanceHour without understanding the billing implications. This policy terminates instances that are closest to the next billing hour, meaning they have been running the longest since the last full hour. While this minimizes wasted partial hours, it can cause frequent terminations of instances that are near the end of their hour, leading to unnecessary churn. In practice, this policy is rarely used unless you have very short-lived workloads.

In production, many teams use the default termination policy because it automatically balances across AZs and considers launch template age. However, they often combine it with lifecycle hooks to gracefully shut down applications. For instance, a web application might use a lifecycle hook to drain connections from the load balancer before termination, preventing dropped requests. The hook waits for the application to finish processing in-flight requests, up to a configurable timeout (e.g., 300 seconds). If the timeout expires, the instance is forcibly terminated.

When misconfigured, termination policies can cause availability issues. For example, if you set NewestInstance and a scale-in event occurs during a blue/green deployment, you might terminate all the new instances, defeating the purpose of the deployment. Similarly, if you forget to configure a termination policy and rely on the default, but your instances have different launch templates (e.g., from a rolling update), the default will terminate the oldest ones, which might be the ones you want to keep. Always test termination behavior in a staging environment before applying to production.