MLS-C01 Modeling Practice Question
A machine learning engineer is training a neural network on Amazon SageMaker using a custom Docker container. The training job fails with an error: 'CUDA out of memory.' The training instance is an ml.p3.2xlarge with 16 GB GPU memory. The model and data fit into memory when using batch size 32, but the engineer wants to maximize GPU utilization. Which approach should the engineer use to fix the out-of-memory error while maintaining efficient training?
⚠ Common exam trap
Watch out — candidates often think mixed precision training (Option A) is the direct solution for CUDA out-of-memory, but it only reduces memory per tensor, not the peak memory from batch size; gradient accumulation is the correct technique to handle large effective batches without exceeding GPU memory.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Implement gradient accumulation with a larger effective batch size
Gradient accumulation allows the engineer to simulate a larger effective batch size by accumulating gradients over multiple forward/backward passes before performing an optimizer step. This keeps the per-step memory footprint low (avoiding CUDA out-of-memory) while maintaining training dynamics similar to a larger batch, thus maximizing GPU utilization without crashing.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Enable mixed precision training
Why it's wrong here
Helps memory but may not fully resolve OOM.
- ✗
Reduce batch size to 1
Why it's wrong here
Underutilizes GPU.
- ✗
Use a CPU-only instance
Why it's wrong here
Much slower for deep learning.
- ✓
Implement gradient accumulation with a larger effective batch size
Why this is correct
Accumulates gradients over smaller batches to simulate larger batches.
Go deeper
Related to this question
About these practice questions
One of 1,672 original MLS-C01 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.