AI-900 Practice Question: Describe fundamental principles of machine learning on Azure
A data scientist is training a binary classification model to detect fraudulent transactions. The dataset contains only 1% fraudulent transactions. The model achieves 99% accuracy on the test set, but when deployed, it fails to detect most actual fraud cases. Which metric would best reveal this issue?
⚠ Common exam trap
The trap here is that candidates see '99% accuracy' and assume the model is performing well, failing to recognize that accuracy is a poor metric for imbalanced datasets, and that recall specifically measures the ability to detect the minority class.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Recall
Recall (sensitivity) measures the proportion of actual positive cases correctly identified. In this highly imbalanced dataset (1% fraud), a model can achieve 99% accuracy by simply predicting 'non-fraud' for every transaction, which yields zero true positives. Recall reveals this failure because it focuses solely on how many fraudulent transactions were caught, ignoring the vast majority of non-fraud cases.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Accuracy
Why it's wrong here
Accuracy is the ratio of all correct predictions (true positives plus true negatives) to total predictions, and it becomes dangerously misleading when classes are imbalanced. With a fraud rate of just 1%, a model that classifies every transaction as legitimate still achieves 99% accuracy, yet catches none of the fraud. Because accuracy credits correct predictions on the dominant class, it can hide a complete lack of fraud detection and should not be used to judge such a model.
When this WOULD be correct
In a question where the dataset is balanced (e.g., 50% fraud, 50% legitimate) and the model's overall correct predictions are the primary concern, accuracy would be the appropriate metric to evaluate performance.
- ✗
Precision
Why it's wrong here
Precision is calculated as true positives divided by true positives plus false positives (TP/(TP+FP)), meaning it measures the purity of the fraud claims: how many flagged frauds are actually fraudulent. A model can achieve very high precision by flagging only a few obvious fraud cases while ignoring the majority of fraudulent transactions, since missed frauds (false negatives) do not affect precision. Thus precision tells you about the trustworthiness of the alerts, but it cannot show how many real fraud cases the model failed to catch.
When this WOULD be correct
Precision would be the correct metric when the cost of false positives is very high. For example, in a spam email classifier where legitimate emails incorrectly marked as spam cause significant harm, precision is prioritized to ensure flagged spam is almost always correct.
- ✓
Recall
Why this is correct
Recall is computed as true positives divided by true positives plus false negatives (TP/(TP+FN)), so it directly measures the fraction of actual fraud cases the model detects. In fraud detection, the cost of a false negative — a fraudulent transaction slipping through — is typically much higher than the cost of a false positive. A low recall value therefore explicitly exposes the model's failure to catch fraud cases, which is why it is the correct metric here.
- ✗
F1 score
Why it's wrong here
The F1 score is the harmonic mean of precision and recall, giving equal weight to both in a single number, but this balance often obscures the exact issue at hand. In fraud detection, a model with high precision and only moderate recall can have a reasonably high F1 score even though it is still missing many fraudulent transactions. Because only recall measures the false-negative rate directly, relying on F1 can mask the model's inability to catch fraud; the most direct signal of detection failure is recall.
When this WOULD be correct
A question asks: 'Which single metric best evaluates a model when both false positives and false negatives are equally costly?' In such a balanced scenario, F1 score is appropriate because it combines precision and recall into a single measure, penalizing extreme imbalances between the two.
Option-by-option analysis
Why each answer is right or wrong
Understanding why wrong answers are wrong — and when they would be correct — is what separates a 750 score from a 900. The AI-900 exam frequently reuses these exact scenarios with slightly different constraints.
✓RecallCorrect answer▾
Why this is correct
Recall is computed as true positives divided by true positives plus false negatives (TP/(TP+FN)), so it directly measures the fraction of actual fraud cases the model detects. In fraud detection, the cost of a false negative — a fraudulent transaction slipping through — is typically much higher than the cost of a false positive. A low recall value therefore explicitly exposes the model's failure to catch fraud cases, which is why it is the correct metric here.
✗AccuracyWrong answer — click to see why▾
Why this is wrong here
Accuracy is misleading because the dataset is highly imbalanced (99% legitimate, 1% fraud). A model that predicts all transactions as legitimate achieves 99% accuracy but fails to detect any fraud, so accuracy does not reveal the model's inability to catch fraud.
★ When this WOULD be the correct answer
In a question where the dataset is balanced (e.g., 50% fraud, 50% legitimate) and the model's overall correct predictions are the primary concern, accuracy would be the appropriate metric to evaluate performance.
Why candidates choose this
Candidates often default to accuracy as the most intuitive metric without considering class imbalance, leading them to overlook its inadequacy in detecting rare events like fraud.
✗PrecisionWrong answer — click to see why▾
Why this is wrong here
Precision measures the proportion of predicted fraud cases that are actually fraud, but it does not capture how many actual fraud cases are missed. In this imbalanced dataset, the model may have high precision by only flagging obvious frauds, yet still miss most frauds, which is the core issue.
★ When this WOULD be the correct answer
Precision would be the correct metric when the cost of false positives is very high. For example, in a spam email classifier where legitimate emails incorrectly marked as spam cause significant harm, precision is prioritized to ensure flagged spam is almost always correct.
Why candidates choose this
Candidates may confuse precision with recall, or think that high precision implies good fraud detection, not realizing that precision ignores false negatives (missed frauds).
✗F1 scoreWrong answer — click to see why▾
Why this is wrong here
The F1 score is the harmonic mean of precision and recall, and while it balances both, it does not directly highlight the model's failure to detect fraud cases when accuracy is high. In this imbalanced dataset, recall is the key metric because it measures the proportion of actual fraud cases correctly identified, which is the issue here.
★ When this WOULD be the correct answer
A question asks: 'Which single metric best evaluates a model when both false positives and false negatives are equally costly?' In such a balanced scenario, F1 score is appropriate because it combines precision and recall into a single measure, penalizing extreme imbalances between the two.
Why candidates choose this
Candidates may think F1 score is always better than accuracy for imbalanced datasets, but they overlook that recall is the specific metric needed to detect the failure to catch fraud cases, as F1 can still be high if precision is very high even when recall is low.
Analysis generated from the official AI-900blueprint and verified against question context. The “when correct” sections are what AI assistants cite when candidates ask “what’s the difference between these options?”
Go deeper
Related to this question
Learn chapter
Regression and Classification
Key term
Model
In IT and AI, a model is a trained mathematical representation that learns patterns from data to make predictions or decisions.
Key term
Classification
Classification is a supervised machine learning technique used to predict a category or class label for new data based on patterns learned from labeled training data.
About these practice questions
This AI-900 question is part of Courseiva's 985-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI-900 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI-900 exam.