MLS-C01 Modeling Practice Question
A data scientist is training a binary classification model for fraud detection. The dataset is highly imbalanced with only 1% fraudulent transactions. The model currently achieves 99% accuracy but only catches 5% of actual fraud cases. Which metric should the data scientist focus on to better evaluate model performance?
⚠ Common exam trap
AWS often tests the misconception that accuracy is always the best metric, but in imbalanced classification, recall or precision-recall curves are more informative, and candidates must recognize that high accuracy can mask poor minority class performance.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Recall
In fraud detection with highly imbalanced data (1% fraud), accuracy is misleading because a model can achieve 99% accuracy by simply predicting 'not fraud' for all transactions. Recall (true positive rate) measures the proportion of actual fraud cases correctly identified, which is critical when the cost of missing fraud is high. The model currently catches only 5% of fraud, so improving recall is the primary goal to reduce false negatives.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Precision
Why it's wrong here
Precision measures the proportion of predicted fraud cases that are actually fraudulent, but with only 1% fraud in the data, even a model that rarely predicts fraud can achieve high precision by avoiding false positives. This fails the scenario because the model already catches only 5% of fraud, and precision ignores the missed fraud cases entirely. It is tempting because precision is commonly used to evaluate classifier quality in balanced datasets, and would be correct when the cost of false positives is extremely high, such as in spam filtering where legitimate emails must not be blocked.
- ✗
Accuracy
Why it's wrong here
Accuracy is high due to class imbalance but does not reflect poor fraud detection.
- ✗
Root Mean Squared Error (RMSE)
Why it's wrong here
RMSE is used for regression tasks, not classification.
- ✓
Recall
Why this is correct
Recall measures the ability to find all positive samples, which is crucial for fraud detection.
Go deeper
Related to this question
About these practice questions
Courseiva writes every MLS-C01 question from scratch — 1,672 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.