MLA-C01 ML Model Development Practice Question
A financial services firm is training a fraud detection model using SageMaker. The dataset is highly imbalanced (0.1% fraudulent transactions). The model currently achieves 99.9% accuracy but only catches 5% of fraud cases. Which metric should the team prioritize to evaluate model performance?
⚠ Common exam trap
MLA-C01 often tests whether candidates default to accuracy or F1 without considering that in highly imbalanced datasets, recall is the metric that directly reflects the model's ability to catch the minority class.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Recall
Recall is the proportion of actual fraud cases that the model correctly identifies, so it directly measures the model's ability to catch fraud. With only 5% of fraud cases caught, recall is extremely low, and improving it is the priority. Accuracy is misleading here because a model that predicts 'not fraud' for every transaction would still achieve 99.9% accuracy on a 0.1% fraud dataset.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Accuracy
Why it's wrong here
Accuracy is misleading on imbalanced data because predicting the majority class yields 99.9% while missing nearly all fraud. Recall, precision or F1 would be correct here. Accuracy suits balanced datasets where class distribution is roughly equal.
- ✗
Precision
Why it's wrong here
Precision measures how many flagged transactions are genuinely fraudulent, so a model catching 5% of fraud can still post high precision by flagging only obvious cases. It is tempting because precision matters when investigation capacity is limited; it would be correct if the priority were minimising false alarms rather than catching fraud.
- ✓
Recall
Why this is correct
Recall measures the proportion of actual fraudulent transactions the model identifies, so it exposes the 5% detection rate that 99.9% accuracy conceals. With 0.1% positives, accuracy is dominated by the majority class, making recall the metric aligned to catching fraud.
- ✗
F1-score
Why it's wrong here
F1-score blends precision and recall into one number, so a model catching only 5% of fraud still scores poorly, but it obscures which dimension is failing. It is tempting because F1 balances both concerns on imbalanced data; it would suit a scenario where false positives and false negatives carry comparable cost.
Go deeper
Related to this question
About these practice questions
This MLA-C01 question is part of Courseiva's 665-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official Amazon Web Services exam blueprint
This MLA-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLA-C01 exam.