MLA-C01 ML Model Development Practice Question
A data scientist trains a binary classification model using SageMaker and obtains an AUC of 0.95 on the test set. However, the precision-recall curve shows low precision for high recall thresholds. The business requires a model that performs well on the minority class. Which metric should the team primarily optimize during hyperparameter tuning?
⚠ Common exam trap
MLA-C01 often tests the misconception that a high AUC-ROC means the model is good for imbalanced data, when in fact AUC can be high while precision at the required recall is unusable — the fix is to tune on F1 or PR-AUC, not AUC.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
F1-score on the validation set
The F1-score is the harmonic mean of precision and recall, so it directly penalizes models that achieve high recall at the cost of low precision — exactly the failure mode described. Since the business cares about the minority class, optimizing F1 during hyperparameter tuning forces the model to balance both false positives and false negatives on that class. AUC-ROC can remain deceptively high (0.95) even when precision collapses at high recall because it aggregates performance across all thresholds and is dominated by the majority class.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Accuracy
Why it's wrong here
Accuracy counts both classes equally, so a model predicting the majority class scores highly while minority-class recall collapses. It suits balanced datasets where class proportions are equal. The stem's minority-class requirement demands precision-recall AUC or F1 instead.
- ✓
F1-score on the validation set
Why this is correct
F1-score balances precision and recall, directly addressing the low-precision-at-high-recall problem and the minority-class requirement. AUC aggregates performance across all thresholds and can look strong despite poor minority-class precision, so tuning against F1 on the validation set targets the constraint the business actually cares about.
- ✗
AUC (Area Under the ROC Curve)
Why it's wrong here
AUC measures overall separability across all classification thresholds, but the precision-recall curve reveals that high recall thresholds yield low precision, meaning the model cannot retrieve the minority class without many false positives. Optimising AUC ignores this trade-off, as it weights true negative rate equally, which is irrelevant when the minority class is the priority. It is tempting because AUC is a standard metric for general model quality, and would be correct if class balance and overall ranking performance were the primary concern.
- ✗
Log loss
Why it's wrong here
Log loss measures probability calibration, not directly precision/recall for the minority class.
Go deeper
Related to this question
About these practice questions
One of 665 original MLA-C01 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
Same concept, more angles
1 more way this is tested on MLA-C01
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A fraud detection model is being trained on imbalanced data. The team wants to ensure the model's precision is optimized. Which objective metric should be used in automatic model tuning?
medium- A.F1
- B.AUC
- ✓ C.Precision
- D.Recall
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official Amazon Web Services exam blueprint
This MLA-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLA-C01 exam.