Overfitting vs Generalization in Regression Models
A data scientist trains a regression model to predict house prices using features like bedrooms, square footage, and location. The model achieves an R-squared of 0.95 on the test set. However, when deployed to predict prices in a new city with different property characteristics, the predictions are very inaccurate. Which concept best explains this poor performance?
Quick Answer
The answer is overfitting, because the model’s high R-squared on the test set masked its inability to generalize to a new city with different property characteristics. Overfitting occurs when a regression model captures noise or dataset-specific patterns—like city-specific price trends—rather than the true underlying relationship, so it fails when faced with data from a different distribution. On the Microsoft Azure AI Fundamentals AI-900 exam, this scenario tests your understanding of the overfitting vs generalization trade-off, often appearing in questions about model evaluation and deployment pitfalls. A common trap is assuming a high R-squared always means a good model; in reality, it can signal overfitting if the test set is too similar to the training data. Remember the mnemonic: “High R-squared, low real-world—overfitting unfurled.”
⚠ Common exam trap
Watch out — candidates often confuse high test-set accuracy with model generalization, failing to recognize that a model can overfit to the test set's distribution and still fail on data from a different domain, which is a core concept of overfitting versus data drift.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Overfitting
The model achieved an R-squared of 0.95 on the test set, indicating excellent performance on data from the same distribution. However, when deployed to a new city with different property characteristics, the predictions were very inaccurate. This is a classic symptom of overfitting, where the model has learned noise and patterns specific to the training data (e.g., city-specific price trends) that do not generalize to unseen data from a different distribution.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Overfitting
Why this is correct
The model performs well on the original test set but fails on data from a different distribution (new city), which is a classic sign of overfitting.
- ✗
Underfitting
Why it's wrong here
Underfitting would cause poor performance on both training and test sets, which is not the case here.
- ✗
High bias
Why it's wrong here
High bias is associated with underfitting, where the model is too simple to capture patterns. This model shows high variance (overfitting).
- ✗
Data drift
Why it's wrong here
Data drift refers to a gradual change in data distribution over time, but the issue here is immediate poor generalization to a different city, not a temporal change.
Go deeper
Related to this question
Learn chapter
Regression and Classification
Key term
Model
In IT and AI, a model is a trained mathematical representation that learns patterns from data to make predictions or decisions.
Key term
Regression
Regression is a type of machine learning algorithm that predicts a continuous numeric output based on input data, used to model relationships between variables.
About these practice questions
One of 985 original AI-900 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
Same concept, more angles
1 more way this is tested on AI-900
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A data scientist trains a regression model to predict house prices using features like bedrooms, square footage, and location. The model achieves a low error on the training data but performs significantly worse when used to predict prices in a new city with different property characteristics. Which concept best explains this poor performance?
medium- A.Underfitting
- ✓ B.Overfitting
- C.Data leakage
- D.Bias-variance tradeoff
Why B: The model performs well on training data but poorly on new data from a different city, which is the classic symptom of overfitting. Overfitting occurs when a model learns noise and specific patterns in the training data that do not generalize to unseen data, especially when the new data has different characteristics (e.g., different property market dynamics). In this case, the model has memorized the training city's price patterns rather than learning generalizable relationships.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI-900 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI-900 exam.