Quick Answer
The answer is to implement gradient accumulation with a larger effective batch size. This technique solves the CUDA out of memory error by splitting a large batch into smaller micro-batches, performing forward and backward passes on each micro-batch to accumulate gradients, and only updating model weights after the full accumulation step. Because the per-step memory footprint remains low—matching the micro-batch size rather than the total effective batch—the GPU memory limit is never exceeded, yet the training dynamics of a larger batch are preserved. On the AWS Certified Machine Learning Specialty MLS-C01 exam, this scenario tests your understanding of memory optimization under fixed GPU constraints, often appearing as a trap where candidates mistakenly choose data parallelism or model parallelism. Remember the mnemonic: “Accumulate gradients, not memory”—you keep the batch small per step but sum the gradients to simulate a larger batch, maximizing utilization without crashing.
This MLS-C01 practice question tests your understanding of modeling. The scenario asks you to isolate a root cause — eliminate options that address a different problem before choosing. After answering, compare your reasoning against the explanation and wrong-answer breakdown below. Once you have made your selection, read the full explanation to reinforce the concept and understand why each distractor is designed to mislead on exam day.
A machine learning engineer is training a neural network on Amazon SageMaker using a custom Docker container. The training job fails with an error: 'CUDA out of memory.' The training instance is an ml.p3.2xlarge with 16 GB GPU memory. The model and data fit into memory when using batch size 32, but the engineer wants to maximize GPU utilization. Which approach should the engineer use to fix the out-of-memory error while maintaining efficient training?
Enable mixed precision training
Why wrong: Helps memory but may not fully resolve OOM.
Reduce batch size to 1
Why wrong: Underutilizes GPU.
Use a CPU-only instance
Why wrong: Much slower for deep learning.
Implement gradient accumulation with a larger effective batch size
Accumulates gradients over smaller batches to simulate larger batches.
Answer choices
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
Implement gradient accumulation with a larger effective batch size
Gradient accumulation allows the engineer to simulate a larger effective batch size by accumulating gradients over multiple forward/backward passes before performing an optimizer step. This keeps the per-step memory footprint low (avoiding CUDA out-of-memory) while maintaining training dynamics similar to a larger batch, thus maximizing GPU utilization without crashing.
Key principle: Answer the scenario, not the keyword: identify the specific constraint before choosing the most familiar-sounding option.
Answer analysis
For each option: why learners choose it and why it is or isn't the right answer here.
Enable mixed precision training
Why it's wrong here
Helps memory but may not fully resolve OOM.
Reduce batch size to 1
Why it's wrong here
Underutilizes GPU.
Use a CPU-only instance
Why it's wrong here
Much slower for deep learning.
Implement gradient accumulation with a larger effective batch size
Why this is correct
Accumulates gradients over smaller batches to simulate larger batches.
Related concept
Read the scenario before looking for a memorised answer.
Common exam traps
The trap here is that candidates may think mixed precision training (Option A) is the direct solution for CUDA out-of-memory, but it only reduces memory per tensor, not the peak memory from batch size; gradient accumulation is the correct technique to handle large effective batches without exceeding GPU memory.
Detailed technical explanation
Under the hood, gradient accumulation works by setting a virtual batch size (e.g., 128) and a per-step micro-batch size (e.g., 32), then accumulating gradients over 4 steps before calling optimizer.step(). This keeps the per-step memory allocation identical to a batch size of 32, avoiding OOM, while the effective batch size of 128 can improve gradient stability and convergence. In practice, this technique is often combined with automatic mixed precision (AMP) to further reduce memory and speed up training, but the core fix for the OOM error is gradient accumulation.
Key takeaway
Answer the scenario, not the keyword: identify the specific constraint before choosing the most familiar-sounding option.
Real-world example
A healthcare organisation deploys an application with a public-facing web tier and a private database tier. The database subnet has no public IP and only accepts connections from the web tier's security group. Questions like this test whether you can design cloud network isolation using VNets/VPCs, subnets, and security group rules.
What to study next
Identify which exam domain this question belongs to, review the core concept, then practise similar questions from the same domain.
Modeling — study guide chapter
Learn the concepts, then practise the questions
Modeling practice questions
Targeted practice on this topic area only
All MLS-C01 questions
1,755 questions across all exam domains
AWS Certified Machine Learning Specialty MLS-C01 study guide
Full concept coverage aligned to exam objectives
MLS-C01 practice test guide
How to use practice tests most effectively before exam day
Related practice questions
Use these pages to review the topic behind this question. This is how one missed question becomes focused revision.
Practise MLS-C01 questions linked to Data Engineering.
Practise MLS-C01 questions linked to Machine Learning Implementation and Operations.
Practise MLS-C01 questions linked to Modeling.
Practise MLS-C01 questions linked to Exploratory Data Analysis.
Practise MLS-C01 questions linked to MLS-C01 fundamentals.
Practise MLS-C01 questions linked to MLS-C01 scenario.
Practise MLS-C01 questions linked to MLS-C01 troubleshooting.
Practice this exam
Short sessions build daily habit. Longer sessions build exam-day stamina. Try a timed session to simulate real conditions.
FAQ
Modeling — This question tests Modeling — Read the scenario before looking for a memorised answer..
The correct answer is: Implement gradient accumulation with a larger effective batch size — Gradient accumulation allows the engineer to simulate a larger effective batch size by accumulating gradients over multiple forward/backward passes before performing an optimizer step. This keeps the per-step memory footprint low (avoiding CUDA out-of-memory) while maintaining training dynamics similar to a larger batch, thus maximizing GPU utilization without crashing.
Identify which exam domain this question belongs to, review the core concept, then practise similar questions from the same domain.
Read the scenario before looking for a memorised answer.
About these practice questions
Courseiva creates original exam-style practice questions with explanations and wrong-answer analysis. It does not publish real exam questions, exam dumps, or protected exam content. Learn why practice questions differ from exam dumps →
Last reviewed: Jun 11, 2026
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.
Question Discussion
Share a tip, memory trick, or ask about the reasoning behind this question. Do not post real exam questions, leaked content, braindumps, or copyrighted exam material. Comments are moderated and may be removed without notice.
Sign in to join the discussion.