Choosing the Right SageMaker Instance for Real-Time Inference
A machine learning engineer is deploying a model to an Amazon SageMaker endpoint for real-time inference. The model is a large ensemble that requires 4 GB of memory. The engineer wants to minimize cost while ensuring the endpoint can handle up to 100 concurrent requests with a latency under 200 ms. Which instance configuration is most appropriate?
Quick Answer
The answer is one ml.c5.xlarge instance with auto-scaling up to 2 instances. This configuration is correct because the ml.c5.xlarge offers 4 vCPUs and 8 GB of memory, comfortably exceeding the 4 GB requirement for the large ensemble model, while its compute-optimized design keeps latency under 200 ms for real-time inference. Auto-scaling to a maximum of two instances ensures the endpoint can handle 100 concurrent requests during peak traffic without over-provisioning, minimizing cost during low-traffic periods by scaling in. On the AWS Certified Machine Learning Specialty MLS-C01 exam, this scenario tests your ability to balance cost and latency when choosing SageMaker instances for real-time inference—a common trap is selecting a larger single instance (like ml.c5.2xlarge) that wastes resources, or a memory-optimized family (like ml.r5) that isn’t needed for compute-bound models. Remember the mnemonic “C for Compute, Scale for Savings” to recall that the c5 family handles compute-heavy inference, and auto-scaling trims costs.
⚠ Common exam trap
A common mix-up: candidates choose a single large instance (like ml.m5.2xlarge) thinking it simplifies management, but auto-scaling with a smaller instance type is more cost-effective and still meets latency requirements under variable load.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
One ml.c5.xlarge instance with auto-scaling up to 2 instances.
The ml.c5.xlarge instance provides sufficient compute (4 vCPUs, 8 GB memory) for the 4 GB model, and auto-scaling up to 2 instances allows handling 100 concurrent requests with low latency while minimizing cost during low traffic. The ml.c5 family is optimized for compute-intensive inference, and auto-scaling ensures the endpoint scales out only when needed, avoiding over-provisioning.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Two ml.t3.medium instances behind a load balancer.
Why it's wrong here
Two ml.t3.medium instances are cheaper per hour than one ml.c5.xlarge, but each t3.medium has only 4 GB memory and 2 vCPUs, which may not handle up to 100 concurrent requests within 200 ms latency. The load balancer adds overhead as well. Cost is lower, but performance is insufficient.
- ✓
One ml.c5.xlarge instance with auto-scaling up to 2 instances.
Why this is correct
ml.c5.xlarge has 4 GB memory, cost-effective, and auto-scaling handles load.
- ✗
One ml.m5.2xlarge instance.
Why it's wrong here
ml.m5.2xlarge has 16 GB memory, more than needed and more expensive.
- ✗
One ml.p3.2xlarge instance.
Why it's wrong here
GPU instance is overkill and costly.
Go deeper
Related to this question
About these practice questions
Courseiva writes every MLS-C01 question from scratch — 1,672 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
Same concept, more angles
2 more ways this is tested on MLS-C01
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A machine learning engineer is using Amazon SageMaker to deploy a model for real-time inference. The model is a large ensemble that requires 4 GB of memory and has a latency requirement of 100 ms. Which instance type and deployment configuration should the engineer choose to optimize cost while meeting requirements?
hard- ✓ A.ml.m5.large (2 vCPU, 8 GB memory)
- B.SageMaker Serverless Inference
- C.ml.c5.large (2 vCPU, 4 GB memory)
- D.ml.p3.2xlarge (8 vCPU, 61 GB memory, 1 GPU)
Why A: ml.m5.large provides 8 GB memory, sufficient for a 4 GB model plus overhead, and is cost-effective for real-time inference with moderate latency requirements. Option B (SageMaker Serverless Inference) is incorrect because cold start latency may exceed the 100 ms requirement. Option C (ml.c5.large) has only 4 GB memory, insufficient for the model. Option D (ml.p3.2xlarge) is GPU-accelerated and expensive, making it overkill for a non-GPU workload.
Variation 2. A machine learning engineer is using Amazon SageMaker to deploy a model for real-time inference. The model must respond within 100 milliseconds. The initial deployment uses a single ml.m5.large instance, but latency is too high. Which change should the engineer make to reduce latency?
easy- ✓ A.Switch to a compute-optimized instance like ml.c5.2xlarge.
- B.Use batch transform instead of real-time endpoint.
- C.Deploy to a single ml.t2.medium instance to reduce cost.
- D.Deploy the model on a multi-model endpoint.
Why A: A compute-optimized instance like ml.c5.2xlarge provides more CPU and memory, reducing inference latency. Option B is wrong because batch transform is for offline predictions, not real-time; it does not reduce latency for real-time inference. Option C is wrong because using a smaller instance (ml.t2.medium) reduces resources and would likely increase latency, not reduce it. Option D is wrong because multi-model endpoints share resources among models and can lead to contention, potentially increasing latency.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.