AI0-001 AI Infrastructure and Technologies Practice Question

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

• ✗

  The learning rate is too high for the larger effective batch size.

  Why it's wrong here

  A high learning rate can cause instability or divergence, but it typically does not reduce throughput; it affects convergence. Throughput is about how many samples are processed per second. The scenario describes a decrease in throughput, not a training failure, so learning rate is unlikely to be the cause.

• ✗

  The model is not using mixed precision training.

  Why it's wrong here

  Mixed precision can improve throughput by using tensor cores, but its absence would affect baseline performance, not necessarily cause a drop when scaling from 8 to 16 GPUs. The issue is specifically about scaling, so communication overhead is a more direct explanation.

• ✗

  Insufficient GPU memory causing out-of-memory errors.

  Why it's wrong here

  Out-of-memory errors would typically cause training to fail rather than just decrease throughput. If memory were insufficient, the batch size would need to be reduced, but the scenario does not mention errors. The issue is more likely related to communication overhead when scaling.

• ✓

  Increased communication overhead for gradient synchronization across GPUs.

  Why this is correct

  In synchronous data parallelism, gradients must be averaged across all GPUs after each backward pass. As the number of GPUs increases, the all-reduce communication cost grows, potentially becoming a bottleneck. This overhead can reduce throughput if the network bandwidth or latency is insufficient, especially when scaling from 8 to 16 GPUs.