Deploying a GPU-Powered Real-Time Endpoint with Multi-AZ High Availability in SageMaker
A data scientist is deploying a real-time inference endpoint using SageMaker. The model is a large NLP model requiring GPU for low latency. The endpoint must be highly available across two Availability Zones. Which deployment configuration meets these requirements?
Quick Answer
This configuration satisfies the two distinct requirements in the scenario by pairing the right instance choice with the right deployment topology. The GPU requirement comes from the model itself, a large NLP model needs the parallel compute and memory bandwidth a GPU instance like ml.p3.2xlarge provides to keep inference latency low, which a CPU-only instance couldn't match for a model of this size. The high-availability requirement is met independently, through deployment topology rather than instance choice: running at least two instances spread across different Availability Zones means that if one AZ becomes unavailable, the endpoint continues serving from the other, and auto-scaling on top of that maintains the desired instance count as load or health conditions change. It's worth noting that the multi-model endpoint capability itself, which lets multiple models share a single endpoint's resources, is a cost and resource-sharing feature, it isn't what provides high availability here. The high availability comes specifically from the multi-instance, multi-AZ deployment and the auto-scaling behavior layered on top of the correctly chosen GPU instance type. Keeping these threads separate is the useful skill: instance type answers the performance requirement, while instance count and AZ placement answer the availability requirement. When a question stacks a performance requirement, like GPU acceleration for a large model, together with an availability requirement, expect the correct configuration to address each requirement with its own distinct piece of the deployment rather than a single setting solving both.
⚠ Common exam trap
The trap here is that candidates may overlook the GPU requirement and choose a cheaper CPU instance (Option A), or confuse batch transform with real-time inference (Option B), or forget that a single instance cannot provide high availability (Option D).
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Deploy a multi-model endpoint on an ml.p3.2xlarge instance with auto-scaling and at least two instances in different AZs
It uses a GPU instance (ml.p3.2xlarge) to meet the low-latency requirement for a large NLP model, and it deploys at least two instances across different Availability Zones (AZs) to achieve high availability. SageMaker multi-model endpoints allow hosting multiple models on the same endpoint, but here the key is the instance type and the multi-instance, multi-AZ deployment for fault tolerance.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Deploy a single model endpoint on an ml.c5.xlarge instance with auto-scaling
Why it's wrong here
CPU instance does not meet GPU requirement.
- ✗
Use SageMaker batch transform on GPU instances
Why it's wrong here
Batch transform is not real-time inference.
- ✓
Deploy a multi-model endpoint on an ml.p3.2xlarge instance with auto-scaling and at least two instances in different AZs
Why this is correct
GPU, auto-scaling, and multi-AZ provide low latency and high availability.
- ✗
Deploy a single model endpoint on an ml.p3.2xlarge instance with one instance
Why it's wrong here
Single instance fails if AZ goes down.
Go deeper
Related to this question
About these practice questions
This MLS-C01 question is part of Courseiva's 1,672-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
Same concept, more angles
1 more way this is tested on MLS-C01
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A data science team is deploying a machine learning model using Amazon SageMaker. The model requires GPU inference and must handle variable traffic with low latency. Which TWO options should the team implement to meet these requirements? (Choose TWO.)
medium- ✓ A.Use a SageMaker multi-model endpoint with a GPU instance to serve multiple models.
- B.Deploy to a SageMaker real-time endpoint using a CPU instance and attach an Elastic Inference accelerator.
- C.Use AWS Lambda with an attached GPU function for inference.
- D.Host the model on a SageMaker batch transform job with GPU instances.
- ✓ E.Deploy the model to a SageMaker real-time endpoint using a GPU instance type.
Why A: A is correct because a SageMaker multi-model endpoint with a GPU instance allows you to host multiple models on a single endpoint, dynamically loading and unloading them based on traffic, while providing GPU acceleration for low-latency inference. This approach efficiently handles variable traffic patterns by scaling the endpoint and leveraging GPU compute for deep learning models.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.