MLS-C01 Modeling Practice Question
A company is building a text classification model to categorize customer support tickets. The dataset is highly imbalanced with 95% of tickets belonging to 'General Inquiry' and 5% to 'Complaint'. The data scientist is using a random forest classifier. Which metric is most appropriate for evaluating model performance on the minority class?
⚠ Common exam trap
Many exam-takers choose accuracy due to its simplicity, failing to recognize that on imbalanced datasets it is a deceptive metric, or they select ROC AUC because it is commonly used for binary classification, but it does not isolate minority class performance as effectively as the F1-score.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
F1-score for 'Complaint'
In a highly imbalanced dataset (95% General Inquiry, 5% Complaint), accuracy is misleading because a model that predicts 'General Inquiry' for every ticket would achieve 95% accuracy but completely fail on the minority class. The F1-score for 'Complaint' is the harmonic mean of precision and recall, providing a balanced evaluation of the model's ability to correctly identify complaints without being skewed by the majority class. For a random forest classifier, this metric directly addresses the minority class performance, which is the primary concern.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Accuracy
Why it's wrong here
Accuracy would be high even if the model predicts only the majority class.
- ✓
F1-score for 'Complaint'
Why this is correct
F1-score balances precision and recall, making it suitable for imbalanced classification.
- ✗
ROC AUC
Why it's wrong here
ROC AUC can be optimistic for imbalanced datasets and does not directly focus on the minority class.
- ✗
Precision for 'Complaint'
Why it's wrong here
Precision alone ignores recall, which could miss many complaints.
Go deeper
Related to this question
About these practice questions
One of 1,672 original MLS-C01 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.