Auto Scaling and Load Balancing work together to keep applications running smoothly even when demand spikes or servers fail. For the DOP-C02 exam, understanding how they create a self-healing, automatically adjusting infrastructure is essential because it moves operations from manual crisis management to automated reliability.
Jump to a section
A simple way to picture Auto Scaling and Load Balancing for Resiliency
A food truck festival is a bustling marketplace of mobile kitchens, each serving a specific menu. The festival is designed to handle the unpredictable ebb and flow of hungry customers throughout the day.
At lunchtime, a massive crowd arrives. The festival organisers have a plan: they don't just let everyone queue at the most popular truck. Instead, they place a 'host' at the entrance who directs customers to whichever truck has the shortest queue. This host is the load balancer, spreading the work evenly so no single truck is overwhelmed while others stand idle. If one truck runs out of ingredients, the host stops sending people there, marking it as 'unhealthy'.
As the afternoon passes, the crowd thins out. The festival has a rule: if a truck's queue drops below three customers for more than ten minutes, that truck is told to shut down early, saving on staff and fuel costs. This is scaling down. But the moment a new lunch rush begins, the host can radio the closed trucks and ask them to fire up their grills again within minutes. This is scaling up. The festival doesn't pay for trucks to stand idle; it only pays for them when they are needed.
Now imagine a storm warning hits the region. The festival operates across multiple city parks. If the storm floods one park, the host instantly redirects all customers to a different park's festival site. This is multi-region failover. The entire system is resilient because the host sees all the trucks across all parks as one giant pool of capacity, automatically routing customers to the food that is available, healthy, and closest to them.
Let us break down these two core concepts. They are often discussed together because they solve complementary problems: one manages the number of servers (capacity), and the other manages the distribution of traffic (routing).
What is Load Balancing?
A load balancer is a traffic cop for your application. Imagine you have a website, and you run it on a single computer (a server). If that computer crashes, your website is gone. If a million people try to visit at once, that one computer gets overwhelmed and slows to a crawl or crashes under the load. A load balancer solves both problems.
You place a load balancer in front of a group of identical servers (called an Auto Scaling Group, which we will cover next). When a user tries to visit your website, their request first hits the load balancer. The load balancer then chooses one server from the pool to handle that specific request. It uses an algorithm to decide which server is best. Common algorithms include 'round robin' (taking turns) and 'least outstanding requests' (sending traffic to the server with the fewest active connections).
The key job of a load balancer is to constantly check whether the servers behind it are healthy. It does this by sending a 'health check' — a simple ping or a request to a specific URL on each server, for example, /health. If a server does not respond within a set time, the load balancer marks it as 'unhealthy' and stops sending it traffic. This is how it provides high availability (HA): if one server dies, the load balancer immediately shifts traffic to the remaining healthy servers, and the users never notice the failure.
What is Auto Scaling?
Auto Scaling is the ability to automatically add or remove servers based on demand. In AWS, this is managed through an Auto Scaling Group (ASG).
An ASG has three core settings: - Minimum size: The smallest number of servers you want running at all times (e.g., 2 servers). - Maximum size: The largest number of servers you will allow (e.g., 20 servers). This prevents runaway costs from a sudden, unexpected spike. - Desired capacity: The ideal number of servers you want under normal conditions (e.g., 2 servers).
The ASG monitors the load on your servers. The most common way to trigger a scaling action is by using a CloudWatch alarm based on metrics like Average CPU Utilisation. For example, you can set a rule: 'If the average CPU across all servers is above 70% for 5 minutes, launch a new server.' This is a scale-out policy. Conversely, if CPU drops below 30% for 10 minutes, terminate one server. This is a scale-in policy.
How They Work Together
This is the critical part for the exam. They are designed to work as an integrated system.
The Auto Scaling Group launches servers. As new servers come online, they are automatically registered with the Load Balancer. The Load Balancer will not send traffic to them until they pass their health check. This prevents users from hitting a server that is still booting up, which would result in an error.
When the Load Balancer detects a server is unhealthy, it stops sending traffic to it. However, it also sends a signal back to the Auto Scaling Group. The ASG sees that the server is marked unhealthy and will terminate it and launch a new replacement server to maintain the desired capacity. This creates a self-healing loop: the Load Balancer detects the failure, the ASG replaces the failed instance.
Why This Replaces Old Methods
Before these tools, operations teams had to manually manage servers. If a website was getting popular, an engineer would have to request new hardware, install the operating system, configure the application, and add it to a load balancer. This took hours or days. If traffic dropped, you were still paying for those unused servers. Auto Scaling and Load Balancing automate this entire lifecycle. They provide elastic scaling — meaning you pay only for what you use, and your application is resilient to failures because the system automatically recovers without human intervention.
Define the Launch Template
Create a blueprint for your servers, specifying the operating system, application code, and server size. This ensures every new server is identical, avoiding configuration drift.
Create the Auto Scaling Group
Configure the ASG with minimum, maximum, and desired capacity. This sets the boundaries for scaling and the normal operating level.
Attach the Load Balancer
Connect the ASG to an Elastic Load Balancer. The ASG automatically registers new instances with the balancer, and the balancer routes traffic to them only after they pass a health check.
Configure Scaling Policies
Set dynamic (target tracking) or scheduled scaling policies. This tells the ASG when to add or remove servers based on metrics like CPU utilisation.
Set Up Health Checks
Define a health check endpoint on your application (e.g., /health). The Load Balancer pings this endpoint periodically to determine if a server is healthy.
Test the System
Simulate a traffic spike to verify the ASG scales out, the Load Balancer distributes traffic, and a server failure is automatically replaced. This validates the entire resilient architecture.
An IT professional managing an e-commerce application for a clothing retailer would use Auto Scaling and Load Balancing constantly. Here is a step-by-step walkthrough of their typical day or week in the real world.
The Scenario: The company runs a web application that sells shoes. They have a major promotion coming up — a 50% off sale on sneakers. They need to ensure the site can handle a sudden 10x increase in traffic without crashing and without paying for too many servers once the sale ends.
Step 1: Configuring the Launch Template The engineer creates a Launch Template. This is a blueprint for every new server. It specifies exactly which Amazon Machine Image (AMI) to use (a pre-configured snapshot of the operating system and application code), what server size (e.g., t3.medium), and how to start the application. This ensures every new server is identical. Without this, manually building servers would inevitably lead to configuration drift — where one server has a slightly different version of the code than another.
Step 2: Setting Up the Auto Scaling Group The engineer creates an ASG using the Launch Template. They set a minimum of 3 servers (for redundancy), a desired capacity of 5 servers (for normal load), and a maximum of 50 servers (to handle the sale). They attach the ASG to a Load Balancer.
Step 3: Configuring Scaling Policies The engineer creates two dynamic scaling policies. The first policy is a 'target tracking' policy: they simply say 'Keep average CPU utilisation at 50%'. The ASG will automatically add servers when CPU goes above 50% and remove them when it goes below 50%. This is the simplest and most common approach. The second policy is a scheduled scaling action: the engineer knows the sale starts at 8 AM on Monday. They schedule the ASG to increase the desired capacity to 30 servers at 7:30 AM so the system has already scaled up before the traffic hits. This avoids the lag between the traffic spike and the scaling response.
Step 4: Monitoring and Reacting On sale day, the engineer monitors CloudWatch dashboards. At 8:05 AM, traffic surges. The average CPU rises to 70%. The target tracking policy kicks in and begins launching new servers. Within 3 minutes, four new servers are online. The Load Balancer registers them, and traffic smoothly distributes across all servers. The CPU stabilises at 50%.
Step 5: Handling a Server Failure
During the sale, one of the 30 servers experiences a hardware fault. The instance becomes unresponsive. The Load Balancer's health check detects that this server is not responding to the /health endpoint. It immediately stops routing traffic there. The Load Balancer sends a notification to the Auto Scaling Group. The ASG sees that the instance is marked unhealthy, terminates it, and launches a new one from the Launch Template. The new server boots up, passes its health check, and is added back into rotation. The entire failure and recovery happens automatically in under 5 minutes, and no customers were affected because the Load Balancer had already moved their connections to other healthy servers.
Step 6: Scaling Down After the sale ends, traffic drops back to normal. The CPU drops below 50%. The target tracking policy starts terminating servers one at a time, respecting a 'cool-down' period to make sure it does not remove servers too quickly. The engineer sees the ASG settle back at 3 servers. They confirm the monthly bill shows they only paid for the extra 40 servers for a few hours, not permanently.
The DOP-C02 exam tests your understanding of the integration between Auto Scaling Groups and Load Balancers, not just the individual services. Here is exactly what to focus on.
Question Types and Traps
The exam loves scenario-based multiple-choice questions where a company describes a problem, and you must choose the solution. A typical question might read: 'A company's application is experiencing high latency during peak hours. The current setup uses a single load balancer and a fixed number of EC2 instances. Which combination of actions should a DevOps engineer take to improve performance and reduce cost?'
The correct answer will include setting up an Auto Scaling Group with a scaling policy based on CPU utilisation and placing it behind an Application Load Balancer.
Common Traps: - The exam will offer a solution that uses only a Load Balancer without an Auto Scaling Group. This is wrong because the Load Balancer cannot fix capacity issues; it can only distribute traffic. If you only add a load balancer to a set of failing servers, you still have too few servers. - The exam might suggest using a 'scale-in' policy when the question asks for a solution to handle increased load. Scale-in removes servers; you need a scale-out policy. - The exam will test the concept of 'cooldown' periods. A common trap is a scaling policy that triggers too fast, causing thrashing (adding and removing servers in a cycle). - The exam tests the difference between ELB types: Classic Load Balancer (older, layer 4), Application Load Balancer (layer 7, HTTP/HTTPS, rules-based routing), and Network Load Balancer (layer 4, ultra-low latency). It often tests that ALB can do path-based routing and host-based routing, which is useful for microservices. - The exam tests the 'termination policy' of an Auto Scaling Group. For example, the default is to terminate the oldest instance first, but there is also a policy to terminate the instance closest to the next billing hour (to save money).
Key Concepts to Memorise: - Lifecycle hooks: These pause an instance during launch or termination so that you can run custom actions (like draining connections or installing software) before the instance goes into service or is terminated. - Suspended processes: You can suspend a process like 'AddToLoadBalancer' or 'HealthCheck' for troubleshooting. The exam will give a scenario where an engineer suspends a process to diagnose an issue. - Warm-up time for scaling: The ASG considers that new instances need time to start serving traffic before they are counted in the scaling metric. This prevents launching too many instances. - Elastic Load Balancing (ELB) connection draining: The amount of time the Load Balancer waits for an in-flight request to complete before terminating a connection to an unhealthy instance. This is critical for ensuring no data loss during a rolling deployment or a failure.
Auto Scaling Groups and Load Balancers work as a pair: the ASG manages capacity, and the Load Balancer distributes traffic across that capacity.
Target tracking policies are the simplest way to scale: set a target metric (like 50% CPU) and the ASG maintains it automatically.
Health checks are mandatory for high availability; a load balancer will not route traffic to an unhealthy server.
Lifecycle hooks allow you to run custom scripts during instance launch or termination, essential for draining connections or pre-warming applications.
Scaling cooldown periods prevent thrashing by adding a delay between scaling actions.
Scheduled scaling is useful for predictable traffic patterns, like a daily morning rush or a planned sale.
Termination policies control which instance is removed first; the default is to remove the oldest instance.
These come up on the exam all the time. Here's how to tell them apart.
Application Load Balancer (ALB)
Operates at Layer 7 (application) of the OSI model, understanding HTTP/HTTPS and WebSocket protocols.
Supports advanced routing rules based on URL path, host header, or query string.
Ideal for microservices architectures where you need to route traffic to different services based on the request URL.
Network Load Balancer (NLB)
Operates at Layer 4 (transport) of the OSI model, handling TCP, UDP, and TLS traffic.
Provides extremely low latency (milliseconds) because it does not inspect the application payload.
Best suited for high-performance, real-time applications like gaming, VoIP, or financial trading.
Target Tracking Scaling
You set a target metric value (e.g., 50% CPU).
The ASG automatically adds or removes instances to maintain that target.
Simpler to configure and less prone to over-scaling because it is a continuous adjustment.
Simple Scaling
You set specific CloudWatch alarms to trigger a specific number of instances to add or remove.
Requires more manual tuning to avoid under- or over-scaling.
Older method, less precise, and can cause thrashing if not tuned properly.
Dynamic Scaling
Responds to real-time load metrics like CPU utilisation or memory.
Handles unpredictable traffic spikes automatically.
No human input needed once the policy is configured.
Scheduled Scaling
Scales based on a fixed time schedule (e.g., every day at 9 AM).
Best for predictable traffic patterns like a daily sales rush.
Requires manual forecasting and configuration of the schedule.
Mistake
Load Balancers increase the capacity of the application by themselves.
Correct
Load Balancers only distribute traffic; they do not add resources. The Auto Scaling Group adds the servers.
Beginners think the balancer is the solution to traffic, not understanding it requires a pool of servers behind it.
Mistake
Auto Scaling means I launch a fixed number of servers and never touch them again.
Correct
Auto Scaling is dynamic — it continuously adds and removes servers based on metrics like CPU or memory.
The word 'Auto' makes people think it is a set-and-forget static size, when it is a dynamic scaling mechanism.
Mistake
Health checks on a Load Balancer are optional.
Correct
Health checks are essential for high availability. Without them, the balancer will send traffic to dead servers, causing errors.
Newcomers set up load balancers without configuring health checks because they do not see their immediate impact in a test environment.
Mistake
If I have an Auto Scaling Group, I do not need a Load Balancer.
Correct
An ASG manages servers, but without a load balancer, traffic would go directly to individual servers, creating a single point of failure and no traffic distribution.
People think scaling IS the solution to everything, forgetting that routing and health detection are separate concerns.
Mistake
Scaling policies are set once and never need changing.
Correct
Scaling policies should be reviewed and adjusted based on changes in application behaviour, traffic patterns, and cost requirements.
Static thinking about infrastructure. Applications evolve, and so must scaling rules.
Mistake
All Load Balancers work the same way.
Correct
There are different types (ALB, NLB, CLB) with different features and performance characteristics. ALB is best for HTTP/HTTPS, NLB for ultra-low latency TCP traffic.
The exam tests specific use cases for each balancer type, so generalising leads to wrong answers.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
The load balancer will stop routing traffic to all servers and return an HTTP 503 error (Service Unavailable) to users. The Auto Scaling Group should detect the failure and launch new servers to recover.
Yes, but it is not recommended for a production web application. Without a load balancer, users would have to connect to individual server IPs, which creates a single point of failure if that one server fails.
Use the AWS Management Console to create an Elastic Load Balancer (choose Application Load Balancer for HTTP traffic), define a listener on port 80 or 443, register a target group, and attach it to an Auto Scaling Group.
A scheduled policy scales at a specific time (e.g., scale up at 8 AM every Monday). A dynamic policy scales in response to real-time metrics like CPU utilisation.
No. The Auto Scaling Group automatically attaches new instances to the load balancer's target group. The load balancer will not send traffic to a new instance until it passes its health check.
A cool-down period is a time window after a scaling action during which the ASG will not launch or terminate additional instances. This prevents rapid, unnecessary scaling actions ('thrashing').
You've finished Auto Scaling and Load Balancing for Resiliency. Continue through the DOP-C02 study guide to build a complete picture of the exam.
Done with this chapter?