SageMaker GPU Inference with Multi-Model Endpoint
A company is deploying a real-time inference endpoint using Amazon SageMaker. The model is a large deep learning model that requires GPU inference. The company wants to minimize latency and cost. Which instance type and deployment strategy should be used?
Quick Answer
The correct choice is to use a real-time endpoint with a GPU instance and enable multi-model endpoints. This deployment strategy minimizes both latency and cost because a multi-model endpoint allows you to host multiple deep learning models on a single GPU instance, sharing the underlying hardware while serving each model in real time. On the AWS Certified Machine Learning Specialty MLS-C01 exam, this question tests your understanding of SageMaker inference options and their trade-offs; a common trap is confusing batch or asynchronous inference with real-time requirements, or assuming Serverless Inference supports GPUs (it does not). Remember that for GPU-accelerated, low-latency serving, you need a real-time endpoint, and multi-model endpoints are the cost-saving mechanism that lets you pack several models onto one GPU instance. A useful memory tip: “Real-time needs GPU, multi-model saves you the money.”
⚠ Common exam trap
Candidates often assume serverless inference (Option A) is always the cheapest and simplest option, but they overlook that serverless does not support GPU instances, making it unsuitable for GPU-required deep learning models.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Use a real-time endpoint with a GPU instance and enable multi-model endpoints.
Using a real-time endpoint with a GPU instance and enabling multi-model endpoints allows the company to serve multiple models on a single GPU instance, reducing cost by sharing the GPU resource while maintaining low latency for real-time inference. Multi-model endpoints load and unload models on demand, minimizing idle GPU time and optimizing cost without sacrificing the low-latency requirement.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Use a serverless inference endpoint with a GPU instance.
Why it's wrong here
Serverless does not support GPUs.
- ✓
Use a real-time endpoint with a GPU instance and enable multi-model endpoints.
Why this is correct
Multi-model endpoints reduce cost by sharing GPU across models.
- ✗
Use a batch transform job with a GPU instance.
Why it's wrong here
Batch transform is not real-time.
- ✗
Use an asynchronous inference endpoint with a GPU instance.
Why it's wrong here
Asynchronous is not real-time.
Go deeper
Related to this question
About these practice questions
One of 1,672 original MLS-C01 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
Same concept, more angles
1 more way this is tested on MLS-C01
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A company is deploying a real-time inference endpoint using Amazon SageMaker. The model is a large deep learning model that requires low latency. The team is concerned about cost. Which SageMaker hosting option should the team use?
medium- A.Use a SageMaker batch transform job.
- B.Use a SageMaker Serverless Inference endpoint.
- C.Use a single-instance endpoint with a large instance type.
- ✓ D.Use a SageMaker multi-model endpoint.
Why D: A SageMaker multi-model endpoint allows you to host multiple models on a single endpoint behind the same serving container, sharing resources and reducing costs while still providing low-latency real-time inference. This is ideal for a large deep learning model that needs low latency but must be cost-effective, as it avoids the expense of dedicated instances for each model.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.