A data scientist is using Amazon SageMaker to train a model. The training job is taking longer than expected. The data scientist notices that the GPU utilization is low. Which action would most likely improve GPU utilization?
Larger batch sizes keep GPU busy.
Why this answer
Low GPU utilization during training often indicates that the GPU is waiting for data to process, a condition known as data bottleneck. Increasing the batch size allows the GPU to process more samples per forward/backward pass, keeping it busier and improving utilization. In SageMaker, this directly impacts the training loop by reducing the frequency of data loading and model update steps.
Exam trap
The trap here is that candidates often assume low GPU utilization means the GPU is underpowered, leading them to choose a larger instance (Option D), when in fact the issue is a data bottleneck that can be mitigated by increasing batch size.
How to eliminate wrong answers
Option A is wrong because switching to a CPU-based instance would likely worsen performance, as CPUs are slower for parallel matrix operations than GPUs. Option C is wrong because decreasing the batch size reduces the amount of work per GPU step, potentially increasing idle time and lowering utilization further. Option D is wrong because using a larger instance type (e.g., more GPUs or faster GPUs) does not address the root cause of low utilization; it may even exacerbate the bottleneck if data loading is the issue.
Option E is wrong because enabling data augmentation adds computational overhead to the data pipeline, which can further slow data delivery and reduce GPU utilization.