MLS-C01 Modeling Practice Question
A company is building a binary classifier to detect fraudulent transactions. The dataset is highly imbalanced with only 0.1% positive cases. The data scientist uses logistic regression and obtains 99.9% accuracy on the test set. Which metric should the data scientist use to evaluate the model's performance?
⚠ Common exam trap
The trap here is that candidates see 'ROC AUC' as a standard metric and forget that it can be inflated by a large number of true negatives in imbalanced datasets, making precision-recall the correct choice for evaluating rare event classifiers.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Precision-recall curve
With only 0.1% positive cases, accuracy is misleading because a model that always predicts 'not fraudulent' achieves 99.9% accuracy. The precision-recall curve focuses on the positive class and is robust to extreme class imbalance, showing the trade-off between precision and recall across thresholds. This makes it the best choice for evaluating a binary classifier on highly imbalanced fraud detection data.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
ROC AUC
Why it's wrong here
ROC AUC can be misleading for highly imbalanced data.
- ✓
Precision-recall curve
Why this is correct
Precision-recall curves focus on the positive class and handle imbalance well.
- ✗
Precision
Why it's wrong here
Precision alone ignores recall.
- ✗
F1 score
Why it's wrong here
F1 score is a single number that may hide performance issues.
Go deeper
Related to this question
About these practice questions
This MLS-C01 question is part of Courseiva's 1,672-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.