Fixing Training Loss Oscillation by Reducing Learning Rate
An AI engineer is training a deep neural network for image recognition. The training loss decreases steadily for the first few epochs but then plateaus and starts to oscillate. Which adjustment is most likely to improve convergence?
Quick Answer
The correct answer is to reduce the learning rate. When training loss decreases steadily then plateaus and begins to oscillate, it typically indicates that the gradient updates are overshooting the minimum of the loss function, a classic symptom of a learning rate that is too high. Reducing the learning rate allows the optimizer to take smaller, more precise steps, damping the oscillations and enabling convergence to a lower loss valley. On the CompTIA AI+ AI0-001 exam, this scenario tests your understanding of hyperparameter tuning and gradient descent dynamics—a common trap is to assume adding more layers or increasing batch size will fix instability, but the primary cause is the step size. Remember the memory tip: “Oscillating loss? Cut the boss”—meaning cut the learning rate when the loss bounces.
⚠ Common exam trap
CompTIA often tests the misconception that increasing the learning rate speeds up convergence, when in fact it causes divergence or oscillation, and that adding layers always improves performance, ignoring the risk of overfitting and optimization difficulty.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Reduce the learning rate
The plateau and oscillation of the training loss indicate that the optimizer is overshooting the minimum due to a learning rate that is too high. Reducing the learning rate allows the optimizer to take smaller, more precise steps, dampening oscillations and enabling convergence to a lower loss. This is a standard technique in gradient descent optimization, often implemented via learning rate schedules or adaptive methods like Adam.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Add more layers
Why it's wrong here
Adding layers may increase model capacity and risk of overfitting, not address oscillation.
- ✗
Increase the learning rate
Why it's wrong here
Increasing learning rate would likely increase oscillation.
- ✗
Increase the batch size
Why it's wrong here
Increasing batch size can reduce variance but may not directly fix oscillation.
- ✓
Reduce the learning rate
Why this is correct
A lower learning rate can smooth convergence and reduce oscillation.
About these practice questions
This AI0-001 question is part of Courseiva's 754-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
Same concept, more angles
1 more way this is tested on AI0-001
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. While training a deep neural network, the loss function fails to converge and oscillates wildly. Which adjustment is most likely to stabilize training?
medium- A.Increase the number of hidden layers
- B.Decrease the batch size
- ✓ C.Reduce the learning rate
- D.Use a test set
Why C: When the loss function oscillates wildly and fails to converge, it typically indicates that the learning rate is too high, causing the optimizer to overshoot the minima. Reducing the learning rate allows the gradient descent updates to take smaller, more stable steps, which helps the loss converge smoothly. This is a fundamental hyperparameter tuning step in deep learning training.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI0-001 practice question is part of Courseiva's free CompTIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI0-001 exam.