My Validation Loss Increases During Fine-Tuning – How Do I Fix Overfitting on Vertex AI?
A team is fine-tuning a large language model on custom data using Vertex AI. They find that the training loss decreases but validation loss increases. What is the best course of action?
Quick Answer
The answer is to reduce the model size or add dropout regularization. This directly addresses the classic overfitting scenario where your training loss decreases but validation loss increases during fine-tuning on Vertex AI, meaning the model is memorizing the training data rather than learning generalizable patterns. Reducing model capacity or introducing dropout forces the model to learn more robust features by limiting its ability to rely on spurious correlations. On the Google Cloud Generative AI Leader exam, this question tests your understanding of regularization techniques in the context of fine-tuning large language models, often appearing as a trap where candidates might mistakenly suggest more data or longer training—which would worsen overfitting. Remember the mnemonic “Drop to Stop”: when validation loss climbs, drop model size or add dropout to stop the overfit.
⚠ Common exam trap
Google Cloud often tests the distinction between underfitting and overfitting, and the trap here is that candidates may confuse increasing validation loss with underfitting and incorrectly choose to increase epochs or learning rate, rather than recognizing the hallmark divergence of overfitting.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Reduce the model size or add dropout regularization.
The increasing validation loss while training loss decreases is a classic sign of overfitting, where the model memorizes the training data but fails to generalize. Reducing model size or adding dropout regularization directly combats overfitting by limiting the model's capacity or introducing noise during training, which forces the model to learn more robust features. This is the best course of action because it addresses the root cause without further exacerbating the problem.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Increase the number of training epochs.
Why it's wrong here
Diverging training and validation loss indicates overfitting, so adding epochs lets the model memorise the custom data further and widens the gap. More epochs suit underfitting, where both losses remain high and the model has not yet captured the underlying pattern.
- ✓
Reduce the model size or add dropout regularization.
Why this is correct
Adding dropout regularisation directly counteracts the overfitting causing validation loss to diverge from training loss. Reducing model capacity likewise limits memorisation of the custom fine-tuning data. Both address the generalisation gap the stem describes, where training loss falls while validation loss rises, restoring alignment between the two curves.
- ✗
Increase the learning rate.
Why it's wrong here
Raising the learning rate makes weight updates larger, which destabilises an already overfitting run and can push validation loss higher still. A higher rate is chosen when training loss plateaus too slowly, not when the model is memorising data and generalisation is degrading.
- ✗
Switch to a smaller batch size.
Why it's wrong here
Shrinking the batch size adds gradient noise and typically worsens the overfitting gap rather than closing it; regularisation, dropout or early stopping address the divergence. Smaller batches are chosen to fit limited GPU memory or to escape sharp minima, not to correct a rising validation loss.
Go deeper
Related to this question
About these practice questions
One of 1,008 original Generative AI Leader practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
Same concept, more angles
2 more ways this is tested on Generative AI Leader
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. Refer to the exhibit. A data scientist is fine-tuning a model. The training loss and accuracy are improving each epoch. However, after training, the model performs poorly on a held-out validation set. What is the most likely issue?
hard- A.Underfitting
- B.Inappropriate learning rate
- C.Data leakage
- ✓ D.Overfitting
Why D: The model's training loss and accuracy improve each epoch, but performance on the validation set is poor. This classic symptom indicates overfitting, where the model memorizes the training data (including noise) rather than learning generalizable patterns. In fine-tuning, this often occurs when the model is trained for too many epochs or the dataset is too small relative to model capacity.
Variation 2. You are a data scientist at a financial institution. You are using Vertex AI to fine-tune a large language model (LLM) for generating financial reports. You have prepared a dataset of 10,000 examples. During fine-tuning, you notice that the training loss is decreasing steadily, but the validation loss is increasing after 5 epochs. The model's generated reports on the validation set contain many factual errors and nonsensical statements. You suspect overfitting. You have limited compute budget and need to improve generalization. What should you do?
hard- A.Increase the learning rate
- B.Increase the number of training epochs to 20
- C.Add more training examples from a public dataset
- ✓ D.Implement early stopping with a patience of 2 epochs
Why D: Early stopping with a patience of 2 epochs is the correct approach because it directly addresses overfitting by halting training when the validation loss fails to improve for a specified number of epochs. This preserves the model's generalization ability without requiring additional compute or data, which aligns with the limited budget constraint. In Vertex AI, early stopping is a built-in hyperparameter tuning strategy that monitors validation metrics and stops the job to prevent further degradation.
You might also like
Similar Generative AI Leader questions
- Which TWO of the following are key differences between generative AI and discriminative AI? (Choose two.)
- Which Google Cloud product provides access to pre-trained foundation models like Gemini?
- A marketing team is using Vertex AI's text generation model to create product descriptions. They want to control the ran…
- A marketing team is using a generative AI model to create ad copy. They want to control the randomness of the output so…
- A team is tuning a large language model for a question-answering task. They notice the model gives high confidence score…
Written by Johnson Ajibi, MSc IT Security
Security Systems Architect · founder of Courseiva
This Generative AI Leader practice question is part of Courseiva's free Google Cloud certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the Generative AI Leader exam.