Selecting a GPU Instance for SageMaker Inference
A data scientist is deploying a model on Amazon SageMaker. The model requires inference on images, and the data scientist wants to use a GPU instance for low latency. However, the data scientist is unsure about the instance type to choose for the endpoint. Which TWO factors should the data scientist consider when selecting the instance type? (Choose TWO.)
Quick Answer
Selecting a GPU instance type for image inference comes down to weighing two very different kinds of constraints: a technical ceiling and a financial one. GPU memory is the technical ceiling, it determines whether the model itself, along with the input images and any batching you want to use, can actually fit and run on the device at all, and running low on GPU memory can force smaller batch sizes or outright failures, both of which hurt throughput and latency. Cost per inference is the financial constraint, and it matters because GPU instances carry meaningfully higher hourly costs than CPU instances, so choosing an instance with more GPU power than the workload needs can meet the latency goal while quietly overspending. Balancing these two factors together is the actual skill being tested: you want enough GPU memory to comfortably host the model and its inputs without constant memory pressure, but not so much headroom that you're paying for capacity the workload will never use. The general requirement for low latency is already given in the scenario rather than being a selection criterion in itself, which is why the two factors that matter here are the concrete technical limit (memory) and the concrete financial limit (cost). When an exam question asks what to weigh when picking a GPU instance type, expect the correct answers to pair a hardware capacity constraint with a cost consideration rather than two purely technical specs.
⚠ Common exam trap
The trap here is that candidates often focus on training-related metrics (like vCPUs or training time) instead of inference-specific factors, or they mistakenly think regional proximity of the S3 bucket affects instance performance, when in fact the endpoint must be in the same Region but the Region itself does not constrain instance type selection.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
The cost per inference for the instance type
Cost per inference directly impacts operational budget, especially with GPU instances that have higher hourly costs; data scientists must balance low latency with cost efficiency. Option E is correct because GPU memory limits the size of models and batch sizes that can be processed in a single inference call, directly affecting latency and throughput. Both factors are critical when selecting an instance type for real-time inference on SageMaker.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
The time taken to train the model
Why it's wrong here
Training time is irrelevant for inference instance selection.
- ✗
The number of vCPUs on the instance
Why it's wrong here
vCPUs are less important for GPU inference.
- ✓
The cost per inference for the instance type
Why this is correct
Cost is a key consideration.
- ✗
The AWS Region of the S3 bucket storing the model
Why it's wrong here
Region does not affect instance type.
- ✓
The GPU memory available on the instance
Why this is correct
GPU memory must fit model and input.
Go deeper
Related to this question
About these practice questions
Courseiva writes every MLS-C01 question from scratch — 1,672 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
Same concept, more angles
1 more way this is tested on MLS-C01
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A data scientist is deploying a model using Amazon SageMaker for real-time inference. The model is memory-intensive and requires a GPU. Which instance type should be selected for the endpoint?
easy- A.i3.2xlarge
- B.c5.2xlarge
- C.r5.2xlarge
- ✓ D.p3.2xlarge
Why D: The p3.2xlarge instance is correct because it provides a GPU (NVIDIA Tesla V100) with high memory bandwidth, which is essential for memory-intensive deep learning models requiring GPU acceleration for real-time inference. SageMaker endpoints for GPU-based models must use instance types from the P or G families, as CPU-only instances like i3, c5, or r5 lack the parallel processing capabilities needed for efficient GPU inference.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.