MLS-C01 Modeling Practice Question
A team deployed a SageMaker endpoint for real-time inference using a PyTorch model. After monitoring, they notice that the latency is highly variable, with p99 latency 10x the p50 latency. The endpoint uses a single ml.c5.2xlarge instance with auto-scaling based on average CPU utilization. Which change is most likely to reduce latency variability?
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Change the auto-scaling metric to 'InvocationsPerInstance'
Scaling based on InvocationsPerInstance allows the endpoint to react more quickly to changes in request volume, reducing the queueing that causes high p99 latency. Option A (increasing batch size) would actually increase latency. Option B (pre-warming) helps with cold starts but not queueing from traffic spikes. Option C (GPU instance) is unlikely to help if the model is CPU-bound and would not address the root cause of latency variability.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Increase the batch size for inference
Why it's wrong here
Larger batches increase per-request latency.
- ✗
Pre-warm the model by sending dummy requests every minute
Why it's wrong here
Warm-up helps cold starts but not the variability due to traffic spikes.
- ✗
Switch to a GPU instance type
Why it's wrong here
GPU may not reduce tail latency if the bottleneck is request queueing.
- ✓
Change the auto-scaling metric to 'InvocationsPerInstance'
Why this is correct
Scaling on invocations per instance prevents overload and reduces queueing.
Go deeper
Related to this question
About these practice questions
One of 1,672 original MLS-C01 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.