MLA-C01 ML Model Development Practice Question
A data scientist trains a binary classification model using SageMaker and obtains an AUC of 0.95 on the test set. However, the precision-recall curve shows low precision for high recall thresholds. The business requires a model that performs well on the minority class. Which metric should the team primarily optimize during hyperparameter tuning?
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
F1-score on the validation set
For imbalanced datasets, the F1-score balances precision and recall, making it a better objective than AUC, which can be misleading when class imbalance exists.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Accuracy
Why it's wrong here
Accuracy is misleading for imbalanced classes; a model predicting majority class always can achieve high accuracy.
- ✓
F1-score on the validation set
Why this is correct
F1 combines precision and recall, directly addressing the minority class performance requirement.
- ✗
AUC (Area Under the ROC Curve)
Why it's wrong here
AUC measures overall separability across all classification thresholds, but the precision-recall curve reveals that high recall thresholds yield low precision, meaning the model cannot retrieve the minority class without many false positives. Optimising AUC ignores this trade-off, as it weights true negative rate equally, which is irrelevant when the minority class is the priority. It is tempting because AUC is a standard metric for general model quality, and would be correct if class balance and overall ranking performance were the primary concern.
- ✗
Log loss
Why it's wrong here
Log loss measures probability calibration, not directly precision/recall for the minority class.
Go deeper
Related to this question
About these practice questions
One of 835 original MLA-C01 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
Same concept, more angles
1 more way this is tested on MLA-C01
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A fraud detection model is being trained on imbalanced data. The team wants to ensure the model's precision is optimized. Which objective metric should be used in automatic model tuning?
medium- A.F1
- B.AUC
- ✓ C.Precision
- D.Recall
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLA-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLA-C01 exam.