A team is training a deep learning model using Horovod distributed training on SageMaker. They observe that the loss stops decreasing after a few epochs. Which technique should they implement to reduce overfitting?
Dropout is a regularization technique that reduces overfitting.
Why this answer
Dropout regularization randomly drops a fraction of neurons during training, which prevents the model from relying too heavily on specific features and forces it to learn more robust representations. This directly addresses overfitting, which is the likely cause of the loss plateauing after a few epochs in a Horovod distributed training setup on SageMaker.
Exam trap
The AIF-C01 exam often tests the misconception that early stopping (reducing epochs) is a regularization technique to reduce overfitting, but the trap here is that early stopping only halts training and does not actively prevent the model from memorizing noise during the epochs it does train.
How to eliminate wrong answers
Option A is wrong because increasing the learning rate can cause the optimizer to overshoot minima, leading to divergence or unstable training, not a reduction in overfitting. Option B is wrong because adding more layers increases model capacity, which typically worsens overfitting by making it easier for the model to memorize noise. Option C is wrong because reducing the number of epochs only stops training earlier; it does not address the underlying overfitting issue, and the loss may still plateau due to memorization rather than convergence.