Overfitting in Machine Learning Models
A data scientist trains a classification model on a dataset of 10,000 labeled emails to distinguish spam from non-spam. The model achieves 99% accuracy on the training data but only 70% accuracy on a held-out test set. Which term best describes this situation?
Quick Answer
The answer is overfitting. This is the correct choice because the model’s high training accuracy of 99% paired with a significantly lower test accuracy of 70% is the classic symptom of overfitting, where the model has memorized noise and specific patterns in the training data instead of learning generalizable features. On the Microsoft Azure AI Fundamentals AI-900 exam, this scenario tests your understanding of model evaluation and the bias-variance tradeoff, often appearing in questions that contrast overfitting with underfitting or proper generalization. A common trap is to confuse overfitting with high variance or to assume high training accuracy always indicates a good model. Remember the memory tip: “If it aces the homework but flunks the final, it’s overfitting.”
⚠ Common exam trap
Many exam-takers confuse 'high training accuracy' with a good model, failing to recognize that the large gap between training and test performance is the hallmark of overfitting, not underfitting or a tradeoff concept.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
B) Overfitting
The model performs exceptionally well on training data (99% accuracy) but poorly on unseen test data (70% accuracy), which is the classic symptom of overfitting. Overfitting occurs when the model learns noise and specific patterns in the training set rather than generalizing to new data, often due to excessive model complexity or insufficient regularization.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
A) Underfitting
Why it's wrong here
Underfitting occurs when the model fails to capture patterns in the training data, resulting in poor performance on both training and test sets, which is not the case here.
- ✓
B) Overfitting
Why this is correct
Overfitting happens when the model memorizes the training data, including noise, leading to high training accuracy but low test accuracy. This matches the described 99% training vs 70% test accuracy.
- ✗
C) Bias-variance tradeoff
Why it's wrong here
Bias-variance tradeoff is a conceptual framework explaining the tension between underfitting and overfitting. It is not the direct term for the observed performance pattern.
- ✗
D) Regularization
Why it's wrong here
Regularization is a technique used to reduce overfitting by penalizing complex models. It is a solution, not the problem described.
Go deeper
Related to this question
Learn chapter
Regression and Classification
Key term
Overfitting
Overfitting occurs when a machine learning model learns the training data too well, including its noise and outliers, causing it to perform poorly on new, unseen data.
Key term
Classification
Classification is a supervised machine learning technique used to predict a category or class label for new data based on patterns learned from labeled training data.
About these practice questions
Courseiva writes every AI-900 question from scratch — 985 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
Same concept, more angles
2 more ways this is tested on AI-900
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A data scientist is training a classification model on a dataset with 100 features and only 500 labeled samples. The model achieves 99% accuracy on the training data but only 68% accuracy on a held-out test set, indicating overfitting. Which technique is most appropriate to directly address this problem?
medium- A.Increase the amount of training data by collecting more samples
- ✓ B.Reduce the number of features used for training
- C.Increase the complexity of the model by adding more layers
- D.Train for more epochs
Why B: Reducing the number of features directly combats overfitting by decreasing model complexity and the risk of learning noise from irrelevant or redundant features. With only 500 samples and 100 features, the model has a high variance problem; feature selection or dimensionality reduction (e.g., using Azure Machine Learning's Filter-Based Feature Selection or PCA) simplifies the hypothesis space, improving generalization to the test set.
Variation 2. A data scientist is developing a classification model to detect fraudulent transactions. The dataset is split into training and test sets. The data scientist repeatedly tunes the model's hyperparameters and evaluates performance on the test set until the test accuracy reaches 95%. However, when the model is deployed on new, unseen data, its accuracy drops to 70%. Which concept best explains this performance degradation?
medium- A.Overfitting to the training data
- B.Data leakage from the training set to the test set
- ✓ C.Overfitting to the test set
- D.Underfitting the training data
Why C: The data scientist repeatedly tuned hyperparameters based on test set performance, effectively using the test set as part of the training process. This causes the model to become specialized to the test set's specific patterns and noise, so it fails to generalize to new, unseen data. This phenomenon is known as overfitting to the test set, where the test set no longer provides an unbiased estimate of real-world performance.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI-900 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI-900 exam.