Automatic Scaling for SageMaker Endpoints
A machine learning engineer is deploying a model using Amazon SageMaker. The model is a PyTorch model that performs real-time inference with low latency requirements. The engineer wants to use automatic scaling based on the number of concurrent requests. Which SageMaker feature should be used to achieve this?
Quick Answer
The answer is to use Amazon SageMaker automatic scaling with a target tracking scaling policy. This is correct because SageMaker’s application auto scaling monitors the SageMakerVariantInvocationsPerInstance metric, which tracks the number of concurrent requests per instance, and automatically adjusts the instance count to maintain a target utilization you define. On the AWS Certified Machine Learning Specialty MLS-C01 exam, this question tests your understanding of how SageMaker handles real-time inference scaling without external components like Elastic Load Balancers or Auto Scaling groups, which are common traps. A key distinction is that SageMaker uses its own built-in load balancer, not an integrated ELB, and target tracking is the native way to scale based on invocations. Memory tip: think “Target Tracking on Invocations Per Instance” — if you see “ELB” or “Auto Scaling group” as options for a SageMaker endpoint, they are almost always distractors.
⚠ Common exam trap
Many exam-takers confuse SageMaker's built-in scaling with generic AWS services like Auto Scaling groups or ELB, not realizing that SageMaker endpoints have their own integrated scaling mechanism via Application Auto Scaling.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Use Amazon SageMaker automatic scaling with a target tracking scaling policy.
Amazon SageMaker automatic scaling with a target tracking scaling policy is the correct feature because it allows the endpoint to dynamically adjust the number of instances based on a predefined metric, such as the number of concurrent requests (e.g., using the SageMakerVariantInvocationsPerInstance metric). This directly meets the requirement for automatic scaling based on concurrent requests while maintaining low latency for real-time PyTorch inference.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Create an AWS Auto Scaling group for the SageMaker endpoint.
Why it's wrong here
SageMaker endpoints do not use Auto Scaling groups.
- ✗
Enable Elastic Load Balancing for the endpoint.
Why it's wrong here
SageMaker endpoints have a built-in load balancer; external ELB is not used for scaling.
- ✓
Use Amazon SageMaker automatic scaling with a target tracking scaling policy.
Why this is correct
This scales based on invocations per instance.
- ✗
Deploy the model behind Amazon API Gateway with a Lambda function.
Why it's wrong here
API Gateway does not manage SageMaker endpoint scaling.
Go deeper
Related to this question
About these practice questions
This MLS-C01 question is part of Courseiva's 1,672-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
Same concept, more angles
1 more way this is tested on MLS-C01
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A team is deploying a machine learning model to production using Amazon SageMaker. They want to automatically scale the endpoint based on the incoming request volume, and they also need to ensure that the endpoint can handle sudden bursts of traffic without dropping requests. Which scaling policy should they use?
medium- A.Scheduled scaling policy for peak hours
- ✓ B.Target tracking scaling policy based on the number of invocations
- C.Simple scaling policy based on average latency
- D.Manual scaling by monitoring CloudWatch alarms
Why B: A target tracking scaling policy based on the number of invocations allows the endpoint to automatically adjust the number of instances to maintain a target metric value (e.g., invocations per instance). This policy can proactively scale out to handle sudden bursts by adding instances before the request queue grows, preventing dropped requests. SageMaker's built-in scaling metric, 'SageMakerVariantInvocationsPerInstance', is ideal for this use case as it directly correlates with traffic volume.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.