AIF-C01 Fundamentals of AI and ML Practice Question

A data scientist is training a model using Amazon SageMaker and notices the training loss is decreasing but validation loss starts increasing after a few epochs. Which technique should they apply to address this?

⚠ Common exam trap

Many candidates confuse overfitting with underfitting or optimization issues, and incorrectly choose to increase learning rate or batch size, not recognizing that rising validation loss with falling training loss is the classic signature of overfitting.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Add regularization (e.g., L1 or L2)

The scenario describes overfitting, where the model memorizes training data but fails to generalize to validation data. Adding regularization (L1 or L2) penalizes large weights, reducing model complexity and improving generalization. This is a standard technique in SageMaker training jobs, often configured via the `regularizer` hyperparameter in frameworks like TensorFlow or MXNet.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Increase batch size
Why it's wrong here
Increasing batch size can speed up training but does not directly address overfitting.
✗
Increase the learning rate
Why it's wrong here
Increasing learning rate might worsen the problem and cause divergence.
✗
Add more training data
Why it's wrong here
More data can help generalize but is not the direct solution for overfitting; regularization is more effective.
✓
Add regularization (e.g., L1 or L2)
Why this is correct
Regularization penalizes large weights and reduces overfitting, which is indicated by increasing validation loss.

Go deeper

AIF-C01 Fundamentals of AI and ML Practice Question

Why each option matters

Option-by-option breakdown

Related to this question