MLS-C01 Modeling Practice Question
A data scientist is building a binary classification model to predict customer churn. The dataset is highly imbalanced, with only 5% of customers churning. The scientist evaluates several models using accuracy, precision, recall, and F1 score. Which metric is most appropriate for comparing model performance in this scenario?
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
F1 score
F1 score is the harmonic mean of precision and recall and is suitable for imbalanced datasets where accuracy can be misleading. Accuracy would be high even if the model predicts no churn ever (95% accuracy). Precision and recall each consider only one aspect, but F1 balances both.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Accuracy
Why it's wrong here
Accuracy is misleading for imbalanced datasets because a model that predicts the majority class always achieves high accuracy.
- ✓
F1 score
Why this is correct
In a highly imbalanced dataset with only 5% churn, accuracy is misleading because a model predicting all non-churn achieves 95% accuracy yet fails entirely at detecting churn. F1 score combines precision and recall via their harmonic mean, penalising extreme imbalance between false positives and false negatives. This directly addresses the constraint of class imbalance, rewarding models that correctly identify the minority churn class without sacrificing precision.
- ✗
Precision
Why it's wrong here
Precision alone does not capture recall; a model with high precision but low recall might miss many churners.
- ✗
Recall
Why it's wrong here
Recall alone does not capture precision; a model with high recall but low precision might have many false positives.
Go deeper
Related to this question
About these practice questions
Courseiva writes every MLS-C01 question from scratch — 1,672 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
Same concept, more angles
1 more way this is tested on MLS-C01
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A company is building a binary classifier to detect fraudulent transactions. The dataset is highly imbalanced (99% legitimate, 1% fraudulent). Which metric is most appropriate for evaluating the model?
medium- A.Accuracy
- B.Mean Squared Error
- ✓ C.F1-score
- D.Area Under the ROC Curve (AUC-ROC)
Why C: Precision and recall (or F1-score) are more informative for imbalanced datasets than accuracy, because a model predicting all legitimate would achieve 99% accuracy but be useless. F1-score balances precision and recall.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.