MLS-C01 Practice Question: Machine Learning Implementation and Operations

You are deploying a PyTorch model to a SageMaker endpoint. The model is large (5 GB) and the endpoint is using an ml.c5.2xlarge instance. Inference latency is higher than required. Which change would most effectively reduce latency?

⚠ Common exam trap

Candidates often choose Elastic Inference (Option C) thinking it provides GPU-like acceleration at lower cost, but they overlook the model size limitation (max ~2 GB) and the added network latency, making it unsuitable for large models like a 5 GB PyTorch model.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Use a GPU instance type such as ml.p3.2xlarge

The primary bottleneck for a large PyTorch model (5 GB) on a CPU instance (ml.c5.2xlarge) is the lack of GPU acceleration for matrix operations and tensor computations. Switching to a GPU instance like ml.p3.2xlarge (with NVIDIA V100 GPUs) offloads the heavy parallel computation to the GPU, drastically reducing per-inference latency for deep learning models, especially those with large parameter counts.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Reduce the batch size in the inference code
Why it's wrong here
Smaller batch may increase overhead.
✗
Decrease the number of model server workers
Why it's wrong here
Fewer workers increase queuing delay.
✗
Enable SageMaker Elastic Inference
Why it's wrong here
Elastic Inference may help but GPU instance often better for large models.
✓
Use a GPU instance type such as ml.p3.2xlarge
Why this is correct
GPU accelerates matrix operations in PyTorch.

Go deeper

MLS-C01 Practice Question: Machine Learning Implementation and Operations

Why each option matters

Option-by-option breakdown

Related to this question