Describe fundamental principles of machine learning on Azure →mediumMultiple ChoiceObjective-mapped

AI-900 Practice Question: Describe fundamental principles of machine learning on Azure

A data scientist trains a machine learning model on a dataset of housing prices. The model achieves 98% accuracy on the training data but only 72% accuracy on a separate test set. What is the most likely problem with this model?

⚠ Common exam trap

Many candidates confuse high training accuracy with a good model, overlooking the critical test accuracy drop that signals overfitting, and may incorrectly select underfitting because they focus only on the low test score.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Overfitting

The model's high accuracy on training data (98%) but significantly lower accuracy on test data (72%) is a classic symptom of overfitting, where the model learns noise and specific patterns in the training set rather than generalizing to new, unseen data. In Azure Machine Learning, this often occurs when the model is too complex (e.g., deep decision trees or high-degree polynomial features) relative to the amount of training data, and regularization techniques like L1/L2 regularization or early stopping are not applied.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Underfitting
Why it's wrong here
Underfitting is the opposite failure mode: the model is too simple to capture the underlying relationship (high bias), so it cannot achieve high accuracy even on the training set itself. Because the model has not learned the relevant patterns, its predictions are similarly poor on both training and test partitions, producing low accuracy everywhere without a widening generalization gap. Therefore a huge gap with strong training performance cannot be explained by underfitting.
When this WOULD be correct
A model achieves low accuracy (e.g., 60%) on both training and test sets, suggesting it is too simple to capture underlying patterns in the data.
✓
Overfitting
Why this is correct
Overfitting occurs when a high-capacity model fits not only the true signal in the training data but also the random noise and idiosyncrasies unique to that sample. The model effectively memorizes training instances (or their exact decision boundaries), so training accuracy is very high; on unseen test data, those memorized patterns do not generalize, yielding much lower test accuracy. This large training-to-test gap is the classic variance-error signature, often cured by regularization, pruning, or obtaining more diverse training data.
✗
Data leakage
Why it's wrong here
Data leakage is a data collection or preprocessing flaw in which information from the test set (or the target variable, or future data) is inadvertently included when fitting the model. Leakage lets the model exploit patterns that will be artificially present in the test features too, which inflates test accuracy—the opposite of the reported low test accuracy. Because the leaked information also helps during training, it can make both training and test look good, so it cannot be the cause of a large drop from training to test.
When this WOULD be correct
A model achieves 99% accuracy on both training and test sets, but fails in production. The question would describe that future data contains a feature (e.g., 'patient ID') that was inadvertently used during training, causing unrealistic performance.
✗
Class imbalance
Why it's wrong here
Class imbalance can degrade performance by causing the model to heavily favor the majority class, especially when accuracy is used as the metric, but it does not directly create a training-versus-test generalization gap. In a skewed dataset, a model may achieve decent overall accuracy by predicting the majority class everywhere, yet this pattern holds similarly on both training and test sets; the minority class suffers poor recall in both. Unless the test set's class distribution differs radically from the training set, imbalance alone would not produce high training accuracy with sharply lower test accuracy.
When this WOULD be correct
A question where a model shows high accuracy on a test set but performs poorly on a specific minority class (e.g., fraud detection with 99% overall accuracy but 0% recall on fraud cases) would make class imbalance the correct answer.

Option-by-option analysis

Why each answer is right or wrong

Understanding why wrong answers are wrong — and when they would be correct — is what separates a 750 score from a 900. The AI-900 exam frequently reuses these exact scenarios with slightly different constraints.

✓OverfittingCorrect answer▾

Why this is correct

Overfitting occurs when a high-capacity model fits not only the true signal in the training data but also the random noise and idiosyncrasies unique to that sample. The model effectively memorizes training instances (or their exact decision boundaries), so training accuracy is very high; on unseen test data, those memorized patterns do not generalize, yielding much lower test accuracy. This large training-to-test gap is the classic variance-error signature, often cured by regularization, pruning, or obtaining more diverse training data.

✗UnderfittingWrong answer — click to see why▾

Why this is wrong here

Underfitting would cause poor performance on both training and test sets, but here training accuracy is high (98%) while test accuracy is low (72%), indicating the model memorized training data rather than failing to learn patterns.

★ When this WOULD be the correct answer

A model achieves low accuracy (e.g., 60%) on both training and test sets, suggesting it is too simple to capture underlying patterns in the data.

Why candidates choose this

Candidates may confuse 'low test accuracy' with underfitting, not realizing that high training accuracy rules out underfitting.

✗Data leakageWrong answer — click to see why▾

Why this is wrong here

Data leakage typically causes overly optimistic performance on both training and test sets, not a large gap between them. Here, the high training accuracy and much lower test accuracy indicate overfitting, not leakage.

★ When this WOULD be the correct answer

A model achieves 99% accuracy on both training and test sets, but fails in production. The question would describe that future data contains a feature (e.g., 'patient ID') that was inadvertently used during training, causing unrealistic performance.

Why candidates choose this

Candidates may confuse the symptoms of overfitting (high training, low test accuracy) with data leakage, as both can lead to misleadingly high training accuracy. They might think the test accuracy drop is due to leakage rather than memorization.

✗Class imbalanceWrong answer — click to see why▾

Why this is wrong here

Class imbalance typically causes poor performance on minority classes, not a large gap between training and test accuracy. The 98% vs 72% discrepancy indicates overfitting, not imbalance.

★ When this WOULD be the correct answer

A question where a model shows high accuracy on a test set but performs poorly on a specific minority class (e.g., fraud detection with 99% overall accuracy but 0% recall on fraud cases) would make class imbalance the correct answer.

Why candidates choose this

Candidates may confuse any performance issue with class imbalance, especially if they recall that imbalance can cause misleading accuracy, but here the key symptom is the train-test gap, not skewed class performance.

Analysis generated from the official AI-900blueprint and verified against question context. The “when correct” sections are what AI assistants cite when candidates ask “what’s the difference between these options?”

Go deeper

AI-900 Practice Question: Describe fundamental principles of machine learning on Azure

Why each option matters

Option-by-option breakdown

Why each answer is right or wrong

Related to this question