Overfitting in Machine Learning — Causes, Detection, and Reduction Techniques
Which THREE of the following are common causes of overfitting in machine learning models?
Quick Answer
The answer is training for too many epochs, as this allows the model to memorize noise and irrelevant patterns in the training data rather than learning generalizable features. This occurs because a complex model, such as a deep neural network with high capacity, can easily overfit when exposed to a small dataset, capturing spurious correlations that do not reflect the true underlying distribution. On the AWS Certified Machine Learning Specialty MLS-C01 exam, this concept tests your understanding of the bias-variance tradeoff and model regularization; a common trap is confusing overfitting with underfitting or assuming more epochs always improve accuracy. To avoid this, remember the mnemonic "Epochs Overfit Noise" (EON) — excessive epochs on limited data lead to memorization, not generalization.
⚠ Common exam trap
The MLS-C01 exam often tests the misconception that more data or regularization causes overfitting, when in fact both are standard countermeasures; the trap is confusing correlation with causation in model training dynamics.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Using a complex model like a deep neural network on a small dataset
A complex model like a deep neural network has high capacity and can easily memorize noise and patterns specific to a small dataset, rather than learning generalizable features. With limited training samples, the model fails to capture the underlying data distribution, leading to poor performance on unseen data.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Using a complex model like a deep neural network on a small dataset
Why this is correct
Complex models on small data overfit.
- ✓
Model has too many parameters relative to the number of training samples
Why this is correct
Too many parameters allow memorization.
- ✗
Having a large dataset with many samples
Why it's wrong here
More data reduces overfitting.
- ✓
Training for too many epochs
Why this is correct
Long training can lead to overfitting.
- ✗
Using regularization techniques
Why it's wrong here
Regularization prevents overfitting.
Go deeper
Related to this question
About these practice questions
This MLS-C01 question is part of Courseiva's 1,672-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
Same concept, more angles
2 more ways this is tested on MLS-C01
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A data scientist is training a neural network for a multi-class classification problem with 100 classes. The model uses a softmax output layer and cross-entropy loss. During training, the loss decreases steadily but the accuracy on the validation set plateaus early. Which of the following is the most likely cause?
hard- A.Batch size is too large
- ✓ B.The model is overfitting the training data
- C.Number of epochs is too small
- D.Learning rate is too high
Why B: When the validation accuracy plateaus early while training loss continues to decrease, it indicates that the model is memorizing the training data rather than learning generalizable patterns. This is classic overfitting, where the softmax output layer produces high-confidence predictions for training samples but fails to generalize to unseen validation data, causing cross-entropy loss to drop on the training set while validation accuracy stagnates.
Variation 2. A data scientist is training a neural network for a multi-class classification problem. The model is overfitting. Which TWO of the following techniques can help reduce overfitting? (Choose two.)
medium- A.Increase the number of hidden layers.
- ✓ B.Add dropout layers after the hidden layers.
- C.Decrease the learning rate.
- ✓ D.Add L2 regularization to the loss function.
- E.Reduce the batch size.
Why B: Dropout layers randomly deactivate a fraction of neurons during training, which prevents the network from relying too heavily on any single neuron and forces it to learn more robust features. This reduces co-adaptation among neurons and is a standard regularization technique to combat overfitting in neural networks. Option D is correct because L2 regularization (also known as weight decay) adds a penalty proportional to the square of the weights to the loss function. This encourages the model to keep weights small, which reduces the complexity of the model and prevents it from fitting noise in the training data. Both dropout and L2 regularization are effective regularization methods for reducing overfitting.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.