Auto-Scaling SageMaker Endpoints
A machine learning engineer is deploying a model to Amazon SageMaker for real-time inference. The model requires low latency and must handle variable traffic patterns. Which SageMaker feature should the engineer use to automatically scale the number of instances based on demand?
Quick Answer
The answer is SageMaker automatic scaling, which is the correct feature for auto-scaling SageMaker endpoints for real-time inference. This capability, powered by Application Auto Scaling, allows you to define scaling policies based on metrics like CPU utilization or request latency, automatically adjusting the number of instances to match variable traffic patterns while maintaining low latency. On the AWS Certified Machine Learning Specialty MLS-C01 exam, this concept tests your understanding of operational efficiency versus manual instance management; a common trap is confusing automatic scaling with multi-model endpoints or Elastic Inference, which address different concerns like model isolation or cost-effective GPU acceleration. Remember that automatic scaling is about dynamic capacity adjustment, not hardware acceleration. A useful memory tip is to think of it as "traffic-responsive instance count"—if traffic spikes, instances scale out; if it drops, they scale in, ensuring you pay only for what you need without sacrificing performance.
⚠ Common exam trap
Many candidates confuse Amazon EC2 Auto Scaling (which scales EC2 instances in an Auto Scaling group) with SageMaker automatic scaling (which scales SageMaker endpoint instances via Application Auto Scaling), leading them to pick B even though it does not directly apply to SageMaker endpoints.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
SageMaker automatic scaling
SageMaker automatic scaling (Application Auto Scaling) is the correct feature because it allows the engineer to define scaling policies (e.g., based on CPU utilization or request latency) that automatically adjust the number of instances behind a SageMaker endpoint in response to real-time traffic patterns. This ensures low latency by maintaining sufficient capacity during spikes and reducing costs during lulls, without manual intervention.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
SageMaker automatic scaling
Why this is correct
SageMaker integrates with Application Auto Scaling to scale the number of instances based on demand.
- ✗
Amazon EC2 Auto Scaling
Why it's wrong here
Amazon EC2 Auto Scaling cannot directly scale SageMaker endpoints; SageMaker uses its own scaling mechanisms.
- ✗
Elastic Inference
Why it's wrong here
Elastic Inference provides GPU acceleration but does not handle automatic scaling of instances.
- ✗
SageMaker Batch Transform
Why it's wrong here
Batch Transform is for offline predictions and does not provide real-time inference or auto scaling.
Go deeper
Related to this question
About these practice questions
Courseiva writes every MLS-C01 question from scratch — 1,672 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
Same concept, more angles
1 more way this is tested on MLS-C01
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A machine learning engineer is deploying a model using Amazon SageMaker and wants to automatically scale the endpoint based on the number of incoming requests. Which scaling policy should be used?
easy- A.Step scaling
- B.Scheduled scaling
- ✓ C.Target tracking scaling
- D.Simple scaling
Why C: Amazon SageMaker endpoints support Application Auto Scaling. A target tracking scaling policy (Option C) is the recommended approach when you want to automatically scale based on a metric like InvocationsPerInstance. It adjusts capacity to maintain the target value of the metric. Step scaling (Option A) requires defining step adjustments and thresholds. Simple scaling is no longer recommended by AWS. Scheduled scaling (Option B) is for predictable traffic patterns. Therefore, Option C is correct.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.