Scaling SageMaker Endpoints for Increased Traffic
A company is using Amazon SageMaker to deploy a model for real-time inference. The model is a deep neural network that requires GPU for low latency. The endpoint currently uses a single ml.p3.2xlarge instance. Traffic is expected to increase by 5x. Which TWO actions should the company take to handle the increased traffic?
Quick Answer
The correct actions are to enable auto-scaling on the endpoint and to switch to a larger GPU instance like ml.p3.8xlarge. Auto-scaling dynamically adjusts the number of instances based on traffic, which is essential for handling a 5x increase in scaling SageMaker endpoint traffic without manual intervention, while upgrading to a larger instance with more GPUs directly increases throughput per request, maintaining low latency for the deep neural network. On the AWS Certified Machine Learning Specialty MLS-C01 exam, this scenario tests your understanding of real-time inference optimization—specifically that GPU instances are non-negotiable for latency-sensitive models, and that scaling policies must be paired with instance sizing. A common trap is assuming adding more instances alone suffices, but without auto-scaling, costs spike and traffic spikes can overwhelm fixed resources. Remember the mnemonic “GPU + Scale” to recall that both compute power and elasticity are required for variable traffic.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Use a larger instance type with more GPUs
The correct actions are A and C. Using a larger instance type with more GPUs (e.g., ml.p3.8xlarge) increases the compute capacity per instance, allowing the model to handle more requests without increasing latency, as GPUs are essential for low-latency inference on deep neural networks. Enabling auto-scaling on the endpoint dynamically adds or removes instances based on traffic, ensuring the endpoint can scale out to handle the 5x increase without manual intervention. Option B is incorrect because switching to CPU would significantly increase latency, as deep neural networks benefit from GPU acceleration. Option D is incorrect because multi-model endpoints are designed to host multiple models on a single instance, not to increase throughput for a single model. Option E is incorrect because decreasing batch size would reduce throughput per request, worsening performance under increased traffic.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Use a larger instance type with more GPUs
Why this is correct
Larger instance provides more GPU compute.
- ✗
Switch to a CPU-based instance
Why it's wrong here
CPU would increase latency for DNN.
- ✓
Enable auto-scaling on the endpoint
Why this is correct
Auto-scaling adjusts capacity based on traffic.
- ✗
Use a multi-model endpoint
Why it's wrong here
Not relevant for single model.
- ✗
Decrease the batch size
Why it's wrong here
Reduces throughput.
Go deeper
Related to this question
About these practice questions
One of 1,672 original MLS-C01 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
Same concept, more angles
2 more ways this is tested on MLS-C01
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A company uses Amazon SageMaker to deploy a real-time inference endpoint for a regression model. The endpoint is experiencing high latency during spikes in traffic. The data scientist needs to reduce latency while maintaining cost efficiency. Which action should the data scientist take?
medium- A.Use batch transform instead of real-time inference
- B.Use a larger instance type for the endpoint
- C.Deploy the model on a multi-model endpoint
- ✓ D.Enable automatic scaling for the endpoint
Why D: Enabling automatic scaling for the SageMaker endpoint allows the number of instances to dynamically adjust based on traffic patterns, reducing latency during spikes by adding capacity when needed and removing it during low traffic to maintain cost efficiency. Automatic scaling uses CloudWatch metrics (e.g., InvocationsPerInstance or CPUUtilization) to trigger scale-out and scale-in policies, ensuring the endpoint can handle bursts without over-provisioning.
Variation 2. A company uses Amazon SageMaker to deploy a model for real-time inference. The endpoint uses an ml.m5.large instance with automatic scaling based on CPU utilization. The team notices that during traffic spikes, the endpoint returns 5xx errors. What should the team do to improve the endpoint's availability?
medium- ✓ A.Increase the instance type to ml.c5.2xlarge.
- B.Reduce the scaling cooldown period.
- C.Place an Application Load Balancer in front of the endpoint.
- D.Use Amazon API Gateway to throttle requests.
Why A: Upgrading the instance type from ml.m5.large to ml.c5.2xlarge provides more CPU and memory resources, which directly addresses the root cause of 5xx errors during traffic spikes — insufficient compute capacity to handle the request load. Automatic scaling based on CPU utilization may not react quickly enough to sudden spikes, leading to request queuing and timeouts that manifest as 5xx errors. A larger instance type increases the baseline throughput, reducing the likelihood of resource exhaustion before scaling can take effect.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.