AI0-001 Machine Learning and Deep Learning Practice Question
A data scientist is building a binary classification model to predict customer churn. The dataset has 10,000 samples with 80% non-churn and 20% churn. The model achieves 95% accuracy but fails to identify churners correctly. Which metric should the scientist focus on to evaluate model performance properly?
⚠ Common exam trap
CompTIA often tests the concept that accuracy is misleading in imbalanced datasets, and candidates mistakenly choose precision or F1-score because they seem more comprehensive, but the question specifically asks for the metric that reveals the model's failure to identify churners, which is recall.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Recall (TPR)
Recall (True Positive Rate) measures the proportion of actual churners correctly identified by the model. With 80% non-churn and 20% churn, a model can achieve 95% accuracy by simply predicting the majority class (non-churn) for all samples, resulting in zero true positives for churn. Recall directly exposes this failure by quantifying how many churners are captured, making it the critical metric for imbalanced classification problems.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Precision
Why it's wrong here
Precision measures how many predicted churners are genuine, but with 80% non-churn the model can score high precision by rarely predicting churn, so it still misses actual churners. Precision suits spam filtering, where false positives are the costly error, not detecting a minority class.
- ✗
F1-score
Why it's wrong here
F1-score is incorrect because it treats false positives and false negatives equally in its calculation, which is not suitable when the cost of one error type significantly outweighs the other. The scenario's emphasis on failing to identify churners correctly indicates a critical need to minimise false negatives, requiring a metric that prioritises recall. F1-score is tempting as it effectively balances precision and recall, making it a good choice for imbalanced datasets where a balanced trade-off between these two error types is desired.
- ✓
Recall (TPR)
Why this is correct
With 80/20 class imbalance, accuracy is misleading because predicting all non-churn yields 80%. Recall measures the proportion of actual churners correctly identified, exposing the model's failure to detect the minority class the business cares about.
- ✗
Specificity
Why it's wrong here
Specificity measures true negatives among actual non-churners, so a model labelling almost everyone as non-churn scores highly while missing churners. It would be relevant if false positives on non-churners mattered; here recall or F1 for the churn class is needed.
About these practice questions
This AI0-001 question is part of Courseiva's 962-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI0-001 practice question is part of Courseiva's free CompTIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI0-001 exam.