mediumMultiple Choice
AIF-C01 Practice Question: A team is building a classifier to detect…
A team is building a classifier to detect fraudulent transactions. The dataset has 99.9% legitimate transactions and 0.1% fraudulent. Which evaluation metric is most appropriate?
⚠ Common exam trap
The AWS AI Practitioner exam often tests the misconception that accuracy is always a good metric, but the trap here is that accuracy is dangerously misleading for imbalanced datasets, and candidates must recognize that AUC-ROC or precision-recall metrics are required for such scenarios.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
AUC-ROC
AUC-ROC is the most appropriate metric for this highly imbalanced classification problem because it evaluates the model's ability to distinguish between fraudulent (positive) and legitimate (negative) transactions across all classification thresholds, without being biased by the overwhelming majority of legitimate transactions. Unlike accuracy, AUC-ROC focuses on the trade-off between true positive rate and false positive rate, making it robust to class imbalance where 99.9% of data is negative.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Mean Absolute Error (MAE)
Why it's wrong here
MAE averages absolute differences between continuous values, making it a regression metric that cannot score discrete fraud labels. It is tempting because it is simple and interpretable, but classification under 0.1% fraud needs precision-recall or PR-AUC to reveal minority-class detection.
- ✗
Root Mean Squared Error (RMSE)
Why it's wrong here
RMSE measures the magnitude of continuous prediction error, so it applies to regression, not classification labels. It is tempting because it penalises large errors, but with 0.1% fraud a classifier's outputs are discrete; precision-recall or PR-AUC captures minority-class detection instead.
- ✓
AUC-ROC
Why this is correct
With only 0.1% fraud, accuracy is misleading because predicting all legitimate scores 99.9%. AUC-ROC measures ranking performance across all thresholds, remaining informative under such extreme class imbalance, so it distinguishes fraudulent from legitimate transactions regardless of the skewed prior.
- ✗
Accuracy
Why it's wrong here
Accuracy counts all correct predictions, so predicting every transaction legitimate scores 99.9% while detecting zero fraud. It is tempting as the intuitive default, but under extreme class imbalance it reflects the majority class only; precision, recall or PR-AUC expose minority-class performance.
Go deeper
Related to this question
About these practice questions
One of 862 original AIF-C01 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AIF-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AIF-C01 exam.