AI-900 Practice Question: Describe fundamental principles of machine learning on Azure
A data scientist trains a machine learning model to predict housing prices. On the training data, the model achieves an R-squared value of 0.99, but on a separate validation dataset it achieves an R-squared of only 0.65. What is the most likely issue with this model?
⚠ Common exam trap
It's easy for candidates to confuse high training accuracy with a good model, overlooking the validation gap, or incorrectly attribute the issue to underfitting or high bias because they focus on the low validation score without considering the training performance.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Overfitting
The model performs exceptionally well on the training data (R² = 0.99) but poorly on the validation data (R² = 0.65), which is a classic symptom of overfitting. Overfitting occurs when the model learns noise and specific patterns in the training set that do not generalize to unseen data, often due to excessive complexity (e.g., too many features or deep decision trees). In Azure Machine Learning, this can be detected by comparing training and validation metrics in automated ML runs or by using regularization techniques like L1/L2 penalties.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Overfitting
Why this is correct
The classic signature of overfitting is a model that achieves very high accuracy on training data but shows a substantial drop on validation or test data. Because the model has enough capacity to memorize the training examples—including their noise and outliers—it fails to generalize to unseen patterns. The large gap between training and validation performance is the defining diagnostic for overfitting.
- ✗
Underfitting
Why it's wrong here
Underfitting occurs when a model is too simplistic to capture the underlying relationships in the data, resulting in low accuracy on both the training and validation sets. Instead of seeing a large gap with training near perfect, an underfit model would show similarly poor performance everywhere. Therefore, the scenario’s high training accuracy directly contradicts underfitting.
When this WOULD be correct
Underfitting would be correct if the model had low R-squared on both training and validation sets (e.g., 0.4 and 0.35), suggesting it is too simple to capture underlying patterns, such as using a linear model for non-linear data.
- ✗
High bias
Why it's wrong here
High bias means the model's assumptions are too rigid, so it systematically misses true patterns and produces high error irrespective of the dataset. This bias leads to underfitting, where even the training error remains high and can never approach near-perfect accuracy. Since the training performance is described as very high, high bias cannot be the cause.
When this WOULD be correct
A question where a model performs poorly on both training and validation data (e.g., R² of 0.4 on training and 0.35 on validation) would indicate high bias/underfitting, often due to an overly simple model like linear regression on nonlinear data.
- ✗
Insufficient training data
Why it's wrong here
Having too few training samples can certainly increase variance and make overfitting more likely, but it is a contributing factor rather than the observed behavior. The evidence—extremely high training performance and much lower validation performance—is the direct symptom of overfitting, and this gap can appear even with a large dataset if the model is too complex or trained too long. Thus, insufficient data is not the best answer because the described pattern is specifically overfitting's signature.
When this WOULD be correct
A model trained on a very small dataset (e.g., 50 samples) shows low R² on both training and validation sets (e.g., 0.4 and 0.3). In that scenario, insufficient training data is the most likely issue.
Option-by-option analysis
Why each answer is right or wrong
Understanding why wrong answers are wrong — and when they would be correct — is what separates a 750 score from a 900. The AI-900 exam frequently reuses these exact scenarios with slightly different constraints.
✓OverfittingCorrect answer▾
Why this is correct
The classic signature of overfitting is a model that achieves very high accuracy on training data but shows a substantial drop on validation or test data. Because the model has enough capacity to memorize the training examples—including their noise and outliers—it fails to generalize to unseen patterns. The large gap between training and validation performance is the defining diagnostic for overfitting.
✗UnderfittingWrong answer — click to see why▾
Why this is wrong here
Underfitting occurs when the model performs poorly on both training and validation data, but here the training R-squared is very high (0.99) while validation is low (0.65), indicating the model memorized training data rather than failing to learn patterns.
★ When this WOULD be the correct answer
Underfitting would be correct if the model had low R-squared on both training and validation sets (e.g., 0.4 and 0.35), suggesting it is too simple to capture underlying patterns, such as using a linear model for non-linear data.
Why candidates choose this
Candidates may confuse poor validation performance with underfitting, not realizing that high training performance rules out underfitting and points to overfitting instead.
✗High biasWrong answer — click to see why▾
Why this is wrong here
High bias (underfitting) would cause poor performance on both training and validation data, not high training R² (0.99) with low validation R² (0.65). The discrepancy indicates overfitting, not bias.
★ When this WOULD be the correct answer
A question where a model performs poorly on both training and validation data (e.g., R² of 0.4 on training and 0.35 on validation) would indicate high bias/underfitting, often due to an overly simple model like linear regression on nonlinear data.
Why candidates choose this
Candidates may confuse 'high bias' with any poor validation performance, not realizing that high bias typically leads to low accuracy everywhere, not just on validation.
✗Insufficient training dataWrong answer — click to see why▾
Why this is wrong here
The model performs well on training data (R²=0.99) but poorly on validation data (R²=0.65), indicating overfitting, not insufficient data. Insufficient data typically causes both training and validation performance to be poor.
★ When this WOULD be the correct answer
A model trained on a very small dataset (e.g., 50 samples) shows low R² on both training and validation sets (e.g., 0.4 and 0.3). In that scenario, insufficient training data is the most likely issue.
Why candidates choose this
Candidates may confuse poor validation performance with a lack of data, not recognizing that the high training R² rules out data insufficiency and points to overfitting.
Analysis generated from the official AI-900blueprint and verified against question context. The “when correct” sections are what AI assistants cite when candidates ask “what’s the difference between these options?”
Go deeper
Related to this question
Learn chapter
Machine Learning Core Concepts
Key term
Model
In IT and AI, a model is a trained mathematical representation that learns patterns from data to make predictions or decisions.
Key term
Validation data
Validation data is a subset of data used during the training of a machine learning model to tune its hyperparameters and evaluate its performance on unseen data before final testing.
About these practice questions
One of 985 original AI-900 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI-900 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI-900 exam.