Overfitting in Machine Learning: Detection and Solutions
Which TWO are valid techniques to reduce overfitting in a deep neural network? (Choose TWO.)
⚠ Common exam trap
CompTIA often tests the distinction between techniques that improve training stability (like gradient clipping or adjusting batch size/learning rate) versus those that directly regularize the model to reduce overfitting (like L2 regularization and dropout), leading candidates to confuse optimization tricks with regularization methods.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
L2 regularization
L2 regularization (option C) is a valid technique to reduce overfitting by adding a penalty term proportional to the square of the weight magnitudes to the loss function. This discourages the network from learning overly complex patterns, effectively shrinking weights and improving generalization. Dropout (option E) randomly drops a fraction of neurons during training, which prevents co-adaptation of features and forces the network to learn more robust representations, also reducing overfitting.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Increase batch size
Why it's wrong here
Batch size changes gradient noise and convergence speed but does not itself constrain model capacity, so it does not directly reduce overfitting. It is tempting because larger batches stabilise training and speed up epochs, and increasing batch size is the correct choice when the goal is throughput or training stability.
- ✗
Increase learning rate
Why it's wrong here
Raising the learning rate changes the size of parameter updates, which can speed convergence or escape shallow minima, but it does not penalise model complexity or limit capacity, so generalisation gaps persist or widen. It is tempting because higher rates sometimes regularise implicitly through noisier steps, yet that is unreliable and not a defined overfitting remedy.
- ✓
L2 regularization
Why this is correct
L2 regularization adds a penalty proportional to the squared magnitude of weights to the loss function, shrinking weights toward zero. This constrains model complexity, reducing variance so the network generalises better rather than memorising training samples, which directly counteracts overfitting.
- ✗
Gradient clipping
Why it's wrong here
Gradient clipping rescales gradients to a threshold, countering exploding gradients in recurrent or deep networks, so it stabilises training rather than reducing overfitting. It is tempting because it genuinely improves convergence on unstable architectures, but it does not constrain model capacity, add regularisation, or penalise complexity the way dropout or weight decay do.
- ✓
Dropout
Why this is correct
Dropout randomly deactivates a proportion of neurons during each training iteration, preventing units from co-adapting to specific training samples. This forces redundant, distributed representations, lowering variance and improving generalisation, which directly reduces overfitting in deep networks.
About these practice questions
This AI0-001 question is part of Courseiva's 962-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
Same concept, more angles
3 more ways this is tested on AI0-001
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A team trained a deep neural network on a limited dataset. The training loss decreases consistently, but the validation loss starts increasing after 20 epochs. What is the most likely issue and the best corrective action?
medium- A.Vanishing gradient; use ReLU activation
- ✓ B.Overfitting; apply regularization like dropout
- C.Underfitting; increase model complexity
- D.Data leakage; reshuffle split
Why B: The training loss decreasing while validation loss increasing after 20 epochs is the classic signature of overfitting: the model has memorized the training data but fails to generalize to unseen data. Applying regularization like dropout forces the network to learn more robust features by randomly dropping neurons during training, reducing overfitting. This is the most direct and effective corrective action for this specific symptom.
Variation 2. A machine learning engineer has a dataset of 100,000 records. She splits it into 70% training, 15% validation, and 15% test sets. After training, the model achieves 95% accuracy on training and 85% on validation. What does the accuracy difference most likely indicate?
easy- A.The validation set is too small
- B.The model generalizes well
- ✓ C.The model is overfitting
- D.The test set should be larger
Why C: The 10% gap between training accuracy (95%) and validation accuracy (85%) is a classic sign of overfitting. The model has memorized patterns specific to the training set rather than learning generalizable features, causing it to perform worse on unseen validation data. In machine learning, a significant drop in performance from training to validation indicates poor generalization, which is the hallmark of overfitting.
Variation 3. A machine learning engineer is preparing to train a deep neural network for image classification. To avoid overfitting, which TWO techniques should the engineer apply? (Select TWO.)
easy- ✓ A.Use dropout regularization.
- ✓ B.Use data augmentation.
- C.Increase the number of layers.
- D.Remove all non-linear activation functions.
- E.Reduce the training dataset size.
Why A: Dropout regularization is a technique that randomly drops a fraction of neurons during training, which prevents the network from relying too heavily on any single neuron and reduces co-adaptation. This acts as a form of ensemble learning and significantly reduces overfitting by improving generalization.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI0-001 practice question is part of Courseiva's free CompTIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI0-001 exam.