My Validation Loss Increases During Fine-Tuning – How Do I Fix Overfitting on Vertex AI?
A team is fine-tuning a large language model on custom data using Vertex AI. They find that the training loss decreases but validation loss increases. What is the best course of action?
Quick Answer
The answer is to reduce the model size or add dropout regularization. This directly addresses the classic overfitting scenario where your training loss decreases but validation loss increases during fine-tuning on Vertex AI, meaning the model is memorizing the training data rather than learning generalizable patterns. Reducing model capacity or introducing dropout forces the model to learn more robust features by limiting its ability to rely on spurious correlations. On the Google Cloud Generative AI Leader exam, this question tests your understanding of regularization techniques in the context of fine-tuning large language models, often appearing as a trap where candidates might mistakenly suggest more data or longer training—which would worsen overfitting. Remember the mnemonic “Drop to Stop”: when validation loss climbs, drop model size or add dropout to stop the overfit.
⚠ Common exam trap
Google Cloud often tests the distinction between underfitting and overfitting, and the trap here is that candidates may confuse increasing validation loss with underfitting and incorrectly choose to increase epochs or learning rate, rather than recognizing the hallmark divergence of overfitting.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Reduce the model size or add dropout regularization.
The increasing validation loss while training loss decreases is a classic sign of overfitting, where the model memorizes the training data but fails to generalize. Reducing model size or adding dropout regularization directly combats overfitting by limiting the model's capacity or introducing noise during training, which forces the model to learn more robust features. This is the best course of action because it addresses the root cause without further exacerbating the problem.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Increase the number of training epochs.
Why it's wrong here
More epochs exacerbate overfitting.
- ✓
Reduce the model size or add dropout regularization.
Why this is correct
Regularization techniques combat overfitting.
- ✗
Increase the learning rate.
Why it's wrong here
Higher learning rate may cause divergence, not solve overfitting.
- ✗
Switch to a smaller batch size.
Why it's wrong here
Batch size affects convergence but not directly overfitting.
Go deeper
Related to this question
About these practice questions
One of 683 original Generative AI Leader practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
Same concept, more angles
2 more ways this is tested on Generative AI Leader
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. Refer to the exhibit. A data scientist is fine-tuning a model. The training loss and accuracy are improving each epoch. However, after training, the model performs poorly on a held-out validation set. What is the most likely issue?
hard- A.Underfitting
- B.Inappropriate learning rate
- C.Data leakage
- ✓ D.Overfitting
Why D: The model's training loss and accuracy improve each epoch, but performance on the validation set is poor. This classic symptom indicates overfitting, where the model memorizes the training data (including noise) rather than learning generalizable patterns. In fine-tuning, this often occurs when the model is trained for too many epochs or the dataset is too small relative to model capacity.
Variation 2. You are a data scientist at a financial institution. You are using Vertex AI to fine-tune a large language model (LLM) for generating financial reports. You have prepared a dataset of 10,000 examples. During fine-tuning, you notice that the training loss is decreasing steadily, but the validation loss is increasing after 5 epochs. The model's generated reports on the validation set contain many factual errors and nonsensical statements. You suspect overfitting. You have limited compute budget and need to improve generalization. What should you do?
hard- A.Increase the learning rate
- B.Increase the number of training epochs to 20
- C.Add more training examples from a public dataset
- ✓ D.Implement early stopping with a patience of 2 epochs
Why D: Early stopping with a patience of 2 epochs is the correct approach because it directly addresses overfitting by halting training when the validation loss fails to improve for a specified number of epochs. This preserves the model's generalization ability without requiring additional compute or data, which aligns with the limited budget constraint. In Vertex AI, early stopping is a built-in hyperparameter tuning strategy that monitors validation metrics and stops the job to prevent further degradation.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This Generative AI Leader practice question is part of Courseiva's free Google Cloud certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the Generative AI Leader exam.