Question 1,666 of 1,672
Using SageMaker Multi-Model Endpoints
A company is deploying a real-time inference endpoint using Amazon SageMaker. The model is a large deep learning model that requires low latency. The team is concerned about cost. Which SageMaker hosting option should the team use?
Quick Answer
The tension in this scenario is between wanting low latency, which usually pushes toward dedicated, well-provisioned infrastructure, and wanting to control cost, which usually pushes toward sharing resources, and a SageMaker multi-model endpoint is designed specifically to resolve that tension. Instead of provisioning a separate, always-on endpoint for every model variant, a multi-model endpoint hosts multiple models behind a single serving container on shared instances, loading models into memory dynamically as they're invoked. This sharing of underlying compute is what drives the cost savings, since the company isn't paying for dedicated infrastructure per model, while the endpoint still serves requests in real time rather than through batch or asynchronous processing, which is what preserves the low-latency behavior the large deep learning model requires. The key idea to take away is that cost-per-model and infrastructure-per-model aren't the same thing, a multi-model endpoint decouples them by letting several models share the same underlying compute footprint instead of each one demanding its own. This makes it a natural fit whenever a scenario emphasizes both real-time responsiveness and cost sensitivity for hosting models, as opposed to a scenario purely about maximizing raw throughput for a single always-hot model. Whenever a question asks how to serve real-time predictions from a large or resource-intensive model while explicitly controlling cost, consider whether a resource-sharing hosting option like a multi-model endpoint fits better than a larger dedicated instance.
⚠ Common exam trap
Test-takers frequently confuse 'low latency' with 'dedicated resources' and choose a single-instance endpoint (Option C), overlooking that multi-model endpoints can achieve low latency through caching and shared infrastructure while significantly reducing cost.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Use a SageMaker multi-model endpoint.
A SageMaker multi-model endpoint allows you to host multiple models on a single endpoint behind the same serving container, sharing resources and reducing costs while still providing low-latency real-time inference. This is ideal for a large deep learning model that needs low latency but must be cost-effective, as it avoids the expense of dedicated instances for each model.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Use a SageMaker batch transform job.
Why it's wrong here
Batch transform is for offline predictions, not real-time.
- ✗
Use a SageMaker Serverless Inference endpoint.
Why it's wrong here
Serverless can have cold start latency.
- ✗
Use a single-instance endpoint with a large instance type.
Why it's wrong here
Single large instance may be underutilized and costly.
- ✓
Use a SageMaker multi-model endpoint.
Why this is correct
Multi-model endpoints share resources and reduce cost per model.
About these practice questions
Courseiva creates original exam-style practice questions with explanations and wrong-answer analysis. It does not publish real exam questions, exam dumps, or protected exam content. Learn why practice questions differ from exam dumps →
Same concept, more angles
1 more way this is tested on MLS-C01
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A company is deploying a real-time inference endpoint using Amazon SageMaker. The model is a large deep learning model that requires GPU inference. The company wants to minimize latency and cost. Which instance type and deployment strategy should be used?
medium- A.Use a serverless inference endpoint with a GPU instance.
- ✓ B.Use a real-time endpoint with a GPU instance and enable multi-model endpoints.
- C.Use a batch transform job with a GPU instance.
- D.Use an asynchronous inference endpoint with a GPU instance.
Why B: Using a real-time endpoint with a GPU instance and enabling multi-model endpoints allows the company to serve multiple models on a single GPU instance, reducing cost by sharing the GPU resource while maintaining low latency for real-time inference. Multi-model endpoints load and unload models on demand, minimizing idle GPU time and optimizing cost without sacrificing the low-latency requirement.
Last reviewed: Jul 4, 2026
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.
Question Discussion
Share a tip, memory trick, or ask about the reasoning behind this question. Do not post real exam questions, leaked content, braindumps, or copyrighted exam material. Comments are moderated and may be removed without notice.
Sign in to join the discussion.