MLA-C01 Practice Question: Using Amazon SageMaker to train a large deep…

A company is using Amazon SageMaker to train a large deep learning model. The training job is taking a very long time. The data scientist suspects that the GPU utilization is low due to inefficient data loading. Which action should the data scientist take to diagnose and address this issue?

⚠ Common exam trap

Candidates often assume adding more GPUs or reducing batch size will speed up training, but without addressing the data pipeline bottleneck, these changes can actually worsen GPU utilization and training time.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Check GPU utilization using Amazon CloudWatch metrics, and if low, optimize the data loading pipeline by using Pipe mode or faster data formats.

Low GPU utilization during deep learning training often indicates a data loading bottleneck, where the GPU spends cycles waiting for data. Amazon CloudWatch provides GPU utilization metrics for SageMaker training jobs, and if utilization is low, optimizing the data pipeline with Pipe mode (streaming data directly from Amazon S3) or using faster data formats like RecordIO or TFRecord can reduce I/O overhead and keep the GPU busy.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Switch to a CPU-only instance to reduce overhead.
Why it's wrong here
CPU-only instances remove the GPUs whose utilisation is being diagnosed, making training far slower and the metric meaningless. It is tempting as a cost-saving or overhead-reduction measure, and would be correct for small models or preprocessing workloads that do not benefit from GPU acceleration.
✓
Check GPU utilization using Amazon CloudWatch metrics, and if low, optimize the data loading pipeline by using Pipe mode or faster data formats.
Why this is correct
CloudWatch exposes GPU utilisation metrics for the training job; sustained low values confirm the bottleneck is data starvation rather than compute. Switching to Pipe mode or optimised formats such as RecordIO or TFRecord raises throughput and GPU utilisation.
✗
Reduce the batch size to speed up training.
Why it's wrong here
Reducing batch size changes gradient statistics and step count; it neither measures nor fixes the input pipeline bottleneck, and can worsen GPU utilisation. It is tempting as a quick memory or speed tweak, and would be correct when the constraint is GPU memory rather than data loading throughput.
✗
Increase the number of GPUs in the training instance.
Why it's wrong here
Adding GPUs leaves the data-loading bottleneck untouched; each GPU still starves, so cost rises while utilisation stays low. It is tempting because more accelerators usually speed training, and would be correct when compute, not input throughput, is the limiting factor.

Quick reference

AWS S3 Storage Class Comparison

Storage Class	Min Duration	Retrieval	Use Case
S3 Standard	None	Immediate	Frequently accessed data
S3 Standard-IA	30 days	Immediate	Infrequent access, rapid retrieval
S3 One Zone-IA	30 days	Immediate	Non-critical infrequent data
S3 Intelligent-Tiering	None	Immediate–hours	Unknown or changing access patterns
S3 Glacier Instant	90 days	Milliseconds	Archive with instant retrieval
S3 Glacier Flexible	90 days	Minutes–hours	Archive, flexible retrieval
S3 Glacier Deep Archive	180 days	Hours	Long-term compliance archive

Go deeper

MLA-C01 Practice Question: Using Amazon SageMaker to train a large deep…

Why each option matters

Option-by-option breakdown

AWS S3 Storage Class Comparison

Related to this question