AIF-C01 Fundamentals of AI and ML Practice Question

A data scientist is training a model using Amazon SageMaker and notices the training loss is decreasing but validation loss starts increasing after a few epochs. Which technique should they apply to address this?

⚠ Common exam trap

Many candidates confuse overfitting with underfitting or optimization issues, and incorrectly choose to increase learning rate or batch size, not recognizing that rising validation loss with falling training loss is the classic signature of overfitting.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Add regularization (e.g., L1 or L2)

The scenario describes overfitting, where the model memorizes training data but fails to generalize to validation data. Adding regularization (L1 or L2) penalizes large weights, reducing model complexity and improving generalization. This is a standard technique in SageMaker training jobs, often configured via the `regularizer` hyperparameter in frameworks like TensorFlow or MXNet.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Increase batch size
Why it's wrong here
Larger batches reduce gradient noise but do not counteract the divergence between training and validation loss, which signals overfitting; regularisation such as dropout or early stopping is required. Increasing batch size is genuinely useful for stabilising gradients and speeding up training on hardware with spare memory, not for closing a generalisation gap.
✗
Increase the learning rate
Why it's wrong here
Raising the learning rate worsens the divergence, since overfitting is driven by the model memorising training data rather than by slow convergence. A higher rate is the right lever when training loss plateaus too slowly or the model underfits, not when validation loss has already begun climbing.
✗
Add more training data
Why it's wrong here
Adding training data can reduce overfitting over time, but it does not directly counter the epoch-by-epoch divergence described, and collecting labelled data is slow. More data is the correct remedy when the model has capacity to spare and the dataset is genuinely too small, not for an immediate regularisation fix.
✓
Add regularization (e.g., L1 or L2)
Why this is correct
Rising validation loss while training loss falls signals overfitting. L1 or L2 regularization penalises large weights, constraining model complexity so it generalises better to unseen data, directly addressing the divergence between the two loss curves.

Go deeper

AIF-C01 Fundamentals of AI and ML Practice Question

Why each option matters

Option-by-option breakdown

Related to this question