Auto Healing and Auto Scaling

Auto Scaling is an AWS service that automatically adjusts the number of EC2 instances in a group based on demand. Auto Healing (or health checks) ensures that unhealthy instances are replaced automatically. Together, they form the foundation for building resilient, highly available applications. The SOA-C02 exam tests your ability to configure, monitor, and troubleshoot these features.

An Auto Scaling group (ASG) is a logical container for EC2 instances that share similar characteristics (e.g., same launch template, VPC, subnets). The ASG maintains a desired capacity, minimum size, and maximum size. The Auto Scaling service constantly monitors the number of healthy instances and compares it to the desired capacity. If the count deviates, it launches or terminates instances to match the desired capacity. Scaling policies (simple, step, or target tracking) modify the desired capacity based on CloudWatch alarms. For example, a target tracking policy might aim for an average CPU utilization of 50%.

Using AWS CLI:

# Create an Auto Scaling group
aws autoscaling create-auto-scaling-group \
    --auto-scaling-group-name my-asg \
    --launch-template LaunchTemplateName=my-template,Version=1 \
    --min-size 1 \
    --max-size 5 \
    --desired-capacity 2 \
    --vpc-zone-identifier subnet-abc,subnet-def

# Attach a target tracking scaling policy
aws autoscaling put-scaling-policy \
    --auto-scaling-group-name my-asg \
    --policy-name cpu50-target \
    --policy-type TargetTrackingScaling \
    --target-tracking-configuration TargetValue=50.0,PredefinedMetricSpecification={PredefinedMetricType=ASGAverageCPUUtilization}

# Describe scaling activities
aws autoscaling describe-scaling-activities --auto-scaling-group-name my-asg

Auto Healing is built into Auto Scaling groups. When an instance fails an EC2 status check (or ELB health check), the ASG marks it as unhealthy and terminates it. Then it launches a new instance to replace it. The termination and launch are separate steps. The ASG does not wait for the instance to recover; it immediately replaces it. This ensures quick recovery.

An ASG can be associated with one or more target groups (ALB/NLB) or Classic Load Balancers. When an instance is launched, it is automatically registered with the load balancer. When terminated, it is deregistered. ELB health checks can be used by the ASG to determine instance health. The ASG also respects the load balancer's connection draining settings.

Lifecycle hooks allow you to pause an instance at launch or termination to perform custom actions. For example, you might want to install software or run a configuration management tool before the instance starts serving traffic. When a lifecycle hook is triggered, the instance enters a 'pending:wait' or 'terminating:wait' state. You must complete the action via the CompleteLifecycleAction API or wait for the timeout. The default timeout is 3600 seconds (1 hour), but you can set it up to 48 hours. You can also send a notification to SNS or SQS to trigger a Lambda function.

An e-commerce company runs a web application on EC2 behind an ALB. During normal hours, traffic is moderate, but during flash sales, traffic spikes 10x. They configure an Auto Scaling group with a target tracking policy on ALB RequestCountPerTarget. The group spans three Availability Zones with a min of 2 and max of 20 instances. They set the health check grace period to 300 seconds to allow instances to warm up. A lifecycle hook at launch runs a script to clear cache and register with a monitoring tool. During a flash sale, the ASG scales out rapidly, adding instances in batches. The cooldown (managed by target tracking) prevents oscillation. The team monitors CloudWatch metrics and sets up alarms for scaling failures. A common issue is that if the max size is too low, the ASG cannot handle the spike, causing throttling. They learned to set max size based on peak load estimates plus a buffer.

A financial services company uses Auto Scaling groups for each microservice. They use lifecycle hooks for graceful shutdown: when an instance is terminating, a hook runs a script to drain connections and flush logs before the instance is terminated. They also use lifecycle hooks to run integration tests on new instances before they are marked healthy. They employ multiple scaling policies: a target tracking policy on CPU and a scheduled scaling policy to increase capacity before known peak times (e.g., market open). They encountered an issue where instances were terminated before draining connections, causing errors. They fixed it by increasing the lifecycle hook timeout and ensuring the draining script completed within the timeout. They also use ELB health checks with a custom path to verify application health. The ASG's health check grace period is set to 600 seconds because the application takes longer to initialize.

A media processing company runs batch jobs on a mix of On-Demand and Spot Instances. They use an Auto Scaling group with a mixed instances policy to diversify across instance types and purchase options. They set a target capacity of 70% Spot and 30% On-Demand. The scaling policy is based on the number of jobs in an SQS queue (custom metric). They use lifecycle hooks to handle Spot interruption: when a Spot instance receives a termination notice, the hook pauses termination and allows the job to checkpoint to S3. They set the lifecycle hook timeout to 120 seconds (Spot notices give 2 minutes). They also use CloudWatch to track Spot instance interruption rates and adjust the mix accordingly. A common mistake is not setting the health check type to 'ELB' when using a load balancer, causing the ASG to rely only on EC2 status checks, which do not detect application failures.