Auto Scaling SageMaker Endpoint Based on CPU Utilization
A company has deployed a machine learning model on Amazon SageMaker for real-time inference. The endpoint uses a single ml.c5.xlarge instance. Recently, the traffic has increased, and the endpoint is returning HTTP 503 (Service Unavailable) errors during peak hours. The CloudWatch metrics show that the CPU utilization is consistently above 90% during peak times, and the Invocations metric shows that requests are being throttled. The data science team has already optimized the model to reduce inference time by 20%, but the errors persist. The company needs to resolve the issue without increasing costs significantly. Which course of action should be taken?
Quick Answer
This scenario is built around a cost constraint as much as a technical one, and that's the detail that should steer you toward auto-scaling rather than any static capacity change. The endpoint is already CPU-saturated during peak hours (above 90%) and throttling requests, so the underlying problem is genuinely insufficient capacity at peak, but because traffic is described as varying with peak hours, adding fixed capacity would mean paying for that extra instance around the clock even during low-traffic periods. Configuring auto-scaling based on CPU utilization solves both halves of the problem at once: it adds instances automatically when CPU climbs during peak demand, which eliminates the throttling and 503 errors, and it scales back down during quieter periods, which keeps costs from rising significantly. The alternatives fail because they either increase cost unconditionally or don't fit the workload: upgrading to a larger instance type raises the baseline cost every hour, even during off-peak times when the larger size isn't needed; batch transform is designed for offline, asynchronous scoring and doesn't serve real-time requests at all; and Spot Instances can be reclaimed by AWS at any time, which risks new service disruptions in a scenario that's already fighting availability problems. Whenever a scenario pairs a capacity shortfall during peak traffic with an explicit constraint to avoid raising costs, dynamic auto-scaling, rather than a bigger or additional fixed instance, is almost always the intended answer.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Configure auto-scaling for the endpoint to add instances based on CPU utilization
Configuring auto-scaling for the endpoint based on CPU utilization dynamically adjusts the number of instances to handle increased traffic, reducing HTTP 503 errors without incurring high costs during low traffic. Option A is wrong because upgrading to a larger instance type (e.g., ml.c5.2xlarge) would increase costs even during low-traffic periods, which does not align with the goal of minimizing cost increases. Option B is wrong because batch transform is designed for offline, asynchronous processing, not real-time inference as required here. Option C is wrong because spot instances can be interrupted and reclaimed by AWS, leading to potential service disruptions, and merely adding more instances without scaling logic does not solve the capacity issue efficiently.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Change the instance type to a larger size, such as ml.c5.2xlarge
Why it's wrong here
Larger instances increase costs even during low traffic and may not be cost-effective.
- ✗
Switch to batch transform to process requests in batches
Why it's wrong here
Batch transform is for offline inference, not real-time.
- ✗
Use spot instances to reduce costs and add more instances
Why it's wrong here
Spot instances can be terminated, causing errors, and do not guarantee capacity.
- ✓
Configure auto-scaling for the endpoint to add instances based on CPU utilization
Why this is correct
Auto-scaling adds instances only when needed, handling peak traffic and reducing costs during low traffic.
Visual reference
Go deeper
Related to this question
About these practice questions
Courseiva writes every MLS-C01 question from scratch — 1,672 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
Same concept, more angles
1 more way this is tested on MLS-C01
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A company is using Amazon SageMaker to deploy a model for real-time inference. The endpoint receives variable traffic and the company wants to optimize cost while maintaining responsiveness. Which scaling policy should be used?
hard- ✓ A.Target tracking scaling based on invocation count
- B.Simple scaling with a cooldown period
- C.Scheduled scaling
- D.Manual scaling
Why A: Target tracking scaling based on invocation count is the correct choice because it automatically adjusts the number of instances in real-time based on a predefined metric (e.g., InvocationsPerInstance), maintaining responsiveness during variable traffic while optimizing cost by scaling down during low demand. This policy uses a target value (e.g., 1000 invocations per instance) and SageMaker Application Auto Scaling continuously monitors CloudWatch metrics to add or remove instances as needed, eliminating manual intervention.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.