Fixing Training Loss Oscillation by Reducing Learning Rate
An AI engineer is training a deep neural network for image recognition. The training loss decreases steadily for the first few epochs but then plateaus and starts to oscillate. Which adjustment is most likely to improve convergence?
Quick Answer
The correct answer is to reduce the learning rate. When training loss decreases steadily then plateaus and begins to oscillate, it typically indicates that the gradient updates are overshooting the minimum of the loss function, a classic symptom of a learning rate that is too high. Reducing the learning rate allows the optimizer to take smaller, more precise steps, damping the oscillations and enabling convergence to a lower loss valley. On the CompTIA AI+ AI0-001 exam, this scenario tests your understanding of hyperparameter tuning and gradient descent dynamics—a common trap is to assume adding more layers or increasing batch size will fix instability, but the primary cause is the step size. Remember the memory tip: “Oscillating loss? Cut the boss”—meaning cut the learning rate when the loss bounces.
⚠ Common exam trap
CompTIA often tests the misconception that increasing the learning rate speeds up convergence, when in fact it causes divergence or oscillation, and that adding layers always improves performance, ignoring the risk of overfitting and optimization difficulty.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Reduce the learning rate
The plateau and oscillation of the training loss indicate that the optimizer is overshooting the minimum due to a learning rate that is too high. Reducing the learning rate allows the optimizer to take smaller, more precise steps, dampening oscillations and enabling convergence to a lower loss. This is a standard technique in gradient descent optimization, often implemented via learning rate schedules or adaptive methods like Adam.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Add more layers
Why it's wrong here
Adding layers increases capacity, which typically worsens oscillation by making the loss surface harder to optimise; the plateau and oscillation indicate the learning rate is too high or the schedule needs decay. More layers would be right when the model underfits and training loss stays high.
- ✗
Increase the learning rate
Why it's wrong here
Raising the learning rate amplifies the oscillation, since the optimiser overshoots the minimum and the loss bounces rather than settling. A lower rate or decay schedule is what restores convergence. Increasing the rate would be correct only when training is progressing far too slowly from an initially tiny value.
- ✗
Increase the batch size
Why it's wrong here
A larger batch reduces gradient noise but does not address the overshooting that causes oscillation, and it can slow convergence per epoch. The plateau and oscillation point to the learning rate being too high. Larger batches would be correct when gradient estimates are too noisy and training is unstable for that reason.
- ✓
Reduce the learning rate
Why this is correct
Oscillating loss after an initial plateau indicates the optimiser is overshooting minima because each update step is too large. Lowering the learning rate shrinks those steps, allowing the network to settle into a smoother convergence path.
About these practice questions
This AI0-001 question is part of Courseiva's 962-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
Same concept, more angles
1 more way this is tested on AI0-001
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. While training a deep neural network, the loss function fails to converge and oscillates wildly. Which adjustment is most likely to stabilize training?
medium- A.Increase the number of hidden layers
- B.Decrease the batch size
- ✓ C.Reduce the learning rate
- D.Use a test set
Why C: When the loss function oscillates wildly and fails to converge, it typically indicates that the learning rate is too high, causing the optimizer to overshoot the minima. Reducing the learning rate allows the gradient descent updates to take smaller, more stable steps, which helps the loss converge smoothly. This is a fundamental hyperparameter tuning step in deep learning training.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI0-001 practice question is part of Courseiva's free CompTIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI0-001 exam.