AIF-C01 Fundamentals of AI and ML Practice Question
A data scientist is training a binary classification model to predict customer churn. The dataset has 10,000 records with 9,500 non-churners and 500 churners. After training a logistic regression model, the model achieves 95% accuracy on the test set. However, the business team reports that the model is not useful because it predicts almost all customers as non-churners. Which metric should the data scientist use to evaluate the model's performance in this scenario?
⚠ Common exam trap
The AIF-C01 exam often tests the misconception that high accuracy always indicates a good model, especially in imbalanced datasets, leading candidates to overlook metrics like recall or precision that better reflect model utility for the specific business problem.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Recall
(Recall) is correct because in this highly imbalanced dataset (95% non-churners vs 5% churners), the model's 95% accuracy is misleading—it can achieve this by simply predicting the majority class (non-churner) for all samples. Recall measures the proportion of actual churners correctly identified (True Positives / (True Positives + False Negatives)), directly addressing the business need to detect churn. A high recall ensures the model captures most churners, even at the cost of some false positives.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Accuracy
Why it's wrong here
Accuracy is not suitable for imbalanced datasets as it can be high even if the model predicts the majority class only.
- ✗
R-squared
Why it's wrong here
R-squared is a metric for regression models, not classification.
- ✗
Precision
Why it's wrong here
Precision measures how many of the predicted churners are actual churners, but it does not reflect how many actual churners were missed.
- ✓
Recall
Why this is correct
Recall measures the proportion of actual churners correctly identified, which is the key metric for this imbalanced problem.
Go deeper
Related to this question
About these practice questions
Courseiva writes every AIF-C01 question from scratch — 619 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AIF-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AIF-C01 exam.