Describe fundamental principles of machine learning on Azure →hardMultiple ChoiceObjective-mapped

AI-900 Practice Question: Describe fundamental principles of machine learning on Azure

A data scientist is building a classification model to detect fraudulent transactions. The dataset has 1,000,000 legitimate transactions and only 1,000 fraudulent ones. The model achieves 99.9% accuracy on the test set, but it fails to catch most fraudulent cases. Which metric should the data scientist prioritize to better evaluate the model's performance on this imbalanced dataset?

⚠ Common exam trap

Test-takers frequently default to accuracy as the universal metric, not recognizing that on imbalanced datasets (like 99.9% majority class), accuracy can be deceptively high while the model fails entirely at its primary task of detecting the minority class.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Recall

Recall measures the proportion of actual positive cases (fraudulent transactions) correctly identified by the model. With only 1,000 fraud cases out of 1,001,000 total transactions, a model that predicts 'legitimate' for every transaction would achieve 99.9% accuracy but 0% recall, making recall the critical metric for imbalanced fraud detection.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Accuracy
Why it's wrong here
Accuracy is the ratio of all correct predictions to total cases, but it becomes misleading when classes are heavily imbalanced, as is typical in transaction fraud. If only 0.1% of transactions are fraudulent, a model that always predicts 'non-fraud' achieves 99.9% accuracy while catching zero fraud cases. Therefore accuracy gives no signal about the model's ability to identify the minority class and should be replaced by recall, precision, and confusion-matrix-based metrics.
When this WOULD be correct
When the dataset is balanced (e.g., equal numbers of positive and negative classes) and the costs of false positives and false negatives are similar, accuracy is a straightforward and appropriate metric for overall model performance.
✗
Mean Squared Error
Why it's wrong here
Mean Squared Error is a loss function for regression, where it averages the squared differences between continuous predicted values and actual numeric targets. Classification outputs are category labels (or probabilities), not ordered numeric values, so calculating squared errors between a label like 'fraud' and 'not fraud' is meaningless. Instead, classification models should be assessed with metrics such as recall, precision, F1-score, or log loss, which operate on the true class/categorical distribution.
When this WOULD be correct
A data scientist builds a regression model to predict house prices. The model's performance is evaluated on a test set, and the goal is to minimize large prediction errors. MSE is the appropriate metric because it penalizes larger errors more heavily.
✓
Recall
Why this is correct
Recall, or true positive rate, is calculated as TP / (TP + FN); in fraud detection this measures the fraction of real fraudulent transactions that the model flags. Because missing a fraudulent transaction creates financial loss and erodes trust, a high recall is the primary business requirement even if it means accepting more false positives. It is therefore the most appropriate evaluation metric for this problem.
✗
R-squared
Why it's wrong here
R-squared (coefficient of determination) measures the proportion of variance in a continuous dependent variable that is explained by a regression model, and it is derived from the sum of squared errors. It assumes an interval/ratio target and a decomposition of total variability; class labels are categorical and have no meaningful variance to explain. While R-squared can be negative when predictions are poor in regression, it is entirely inapplicable to classification and gives no information about fraud detection performance.
When this WOULD be correct
In a regression problem where the goal is to evaluate how well a linear model fits the data, such as predicting house prices based on features, R-squared would be the appropriate metric to assess model performance.

Option-by-option analysis

Why each answer is right or wrong

Understanding why wrong answers are wrong — and when they would be correct — is what separates a 750 score from a 900. The AI-900 exam frequently reuses these exact scenarios with slightly different constraints.

✓RecallCorrect answer▾

Why this is correct

Recall, or true positive rate, is calculated as TP / (TP + FN); in fraud detection this measures the fraction of real fraudulent transactions that the model flags. Because missing a fraudulent transaction creates financial loss and erodes trust, a high recall is the primary business requirement even if it means accepting more false positives. It is therefore the most appropriate evaluation metric for this problem.

✗AccuracyWrong answer — click to see why▾

Why this is wrong here

Accuracy is misleading for imbalanced datasets because a model can achieve high accuracy by simply predicting the majority class (legitimate transactions), while failing to detect the minority class (fraud). Here, 99.9% accuracy can occur even if the model never predicts fraud.

★ When this WOULD be the correct answer

When the dataset is balanced (e.g., equal numbers of positive and negative classes) and the costs of false positives and false negatives are similar, accuracy is a straightforward and appropriate metric for overall model performance.

Why candidates choose this

Candidates often default to accuracy as the primary metric because it is intuitive and commonly used, without considering the impact of class imbalance on its validity.

✗Mean Squared ErrorWrong answer — click to see why▾

Why this is wrong here

Mean Squared Error (MSE) is a regression metric, not suitable for classification tasks like fraud detection. It measures average squared difference between predicted and actual values, which doesn't apply to binary outcomes.

★ When this WOULD be the correct answer

A data scientist builds a regression model to predict house prices. The model's performance is evaluated on a test set, and the goal is to minimize large prediction errors. MSE is the appropriate metric because it penalizes larger errors more heavily.

Why candidates choose this

Candidates may confuse MSE as a general error metric applicable to any model, not realizing it is specific to regression and inappropriate for classification with imbalanced classes.

✗R-squaredWrong answer — click to see why▾

Why this is wrong here

R-squared is a metric for regression models, measuring the proportion of variance explained by the model. It is not applicable to classification tasks like fraud detection.

★ When this WOULD be the correct answer

In a regression problem where the goal is to evaluate how well a linear model fits the data, such as predicting house prices based on features, R-squared would be the appropriate metric to assess model performance.

Why candidates choose this

Candidates may confuse R-squared with a classification metric or think it measures overall model fit regardless of task, especially if they have limited experience with evaluation metrics.

Analysis generated from the official AI-900blueprint and verified against question context. The “when correct” sections are what AI assistants cite when candidates ask “what’s the difference between these options?”

Go deeper

AI-900 Practice Question: Describe fundamental principles of machine learning on Azure

Why each option matters

Option-by-option breakdown

Why each answer is right or wrong

Related to this question