MLA-C01 ML Model Development Practice Question
A company is using SageMaker to train a large model using data parallelism with the SageMaker distributed data parallelism library. They notice that the training throughput is not scaling linearly with the number of GPUs. Which THREE factors could be causing this?
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
I/O bottleneck from reading data from Amazon S3
Communication overhead from gradient synchronization, I/O bottlenecks from reading data, and an inefficient loss scaling strategy can all limit scaling. Model size alone is not a scaling issue if it fits on GPUs. Instance type differences affect speed but not scaling linearity directly.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
I/O bottleneck from reading data from Amazon S3
Why this is correct
Slow data loading can starve GPUs, reducing scaling efficiency.
- ✗
Using different instance types across the cluster
Why it's wrong here
Different instance types may cause stragglers but it's not a typical cause for non-linear scaling in a homogeneous cluster.
- ✗
Model size too large for the GPUs
Why it's wrong here
If the model doesn't fit, data parallelism wouldn't work at all; this is a model parallelism issue.
- ✓
Inefficient loss scaling strategy
Why this is correct
Loss scaling with mixed precision can affect convergence and training dynamics, potentially impacting scaling.
- ✓
Communication overhead from gradient synchronization
Why this is correct
Allreduce operations can become a bottleneck as the number of GPUs increases.
Go deeper
Related to this question
About these practice questions
One of 835 original MLA-C01 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLA-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLA-C01 exam.