The answer is recall, because when evaluating classification models on imbalanced data, accuracy becomes dangerously misleading—a model that simply predicts the majority class for every instance can still achieve 95% accuracy while failing to identify any churners. Recall directly measures the proportion of actual positive cases (churners) that the model correctly captures, calculated as true positives divided by the sum of true positives and false negatives, which is exactly what the business needs to detect churn. On the AWS Certified AI Practitioner AIF-C01 exam, this scenario tests your understanding that accuracy is not the default metric for imbalanced datasets; the common trap is to assume high accuracy means good performance, when in fact the model is useless for the minority class. A reliable memory tip: recall is about “catching the rare ones”—think of it as the model’s ability to recall all the churners from the dataset, even if it means some false alarms.
AIF-C01 Fundamentals of AI and ML Practice Question
This AIF-C01 practice question tests your understanding of fundamentals of ai and ml. Read the scenario carefully and evaluate each option against the stated constraints before committing to an answer. After answering, compare your reasoning against the explanation and wrong-answer breakdown below. Once you have made your selection, read the full explanation to reinforce the concept and understand why each distractor is designed to mislead on exam day.
A data scientist is training a binary classification model to predict customer churn. The dataset has 10,000 records with 9,500 non-churners and 500 churners. After training a logistic regression model, the model achieves 95% accuracy on the test set. However, the business team reports that the model is not useful because it predicts almost all customers as non-churners. Which metric should the data scientist use to evaluate the model's performance in this scenario?
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Recall
Option D (Recall) is correct because in this highly imbalanced dataset (95% non-churners vs 5% churners), the model's 95% accuracy is misleading—it can achieve this by simply predicting the majority class (non-churner) for all samples. Recall measures the proportion of actual churners correctly identified (True Positives / (True Positives + False Negatives)), directly addressing the business need to detect churn. A high recall ensures the model captures most churners, even at the cost of some false positives.
Key principle: Answer the scenario, not the keyword: identify the specific constraint before choosing the most familiar-sounding option.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
✗
Accuracy
Why it's wrong here
Accuracy is not suitable for imbalanced datasets as it can be high even if the model predicts the majority class only.
✗
R-squared
Why it's wrong here
R-squared is a metric for regression models, not classification.
✗
Precision
Why it's wrong here
Precision measures how many of the predicted churners are actual churners, but it does not reflect how many actual churners were missed.
✓
Recall
Why this is correct
Recall measures the proportion of actual churners correctly identified, which is the key metric for this imbalanced problem.
Related concept
Read the scenario before looking for a memorised answer.
Common exam traps
Common exam trap: answer the scenario, not the keyword
Cisco often tests the misconception that high accuracy always indicates a good model, especially in imbalanced datasets, leading candidates to overlook metrics like recall or precision that better reflect model utility for the specific business problem.
Detailed technical explanation
How to think about this question
Recall is also known as sensitivity or True Positive Rate (TPR). In imbalanced classification, threshold tuning (e.g., lowering the decision threshold from 0.5 to 0.3) can trade precision for recall, and metrics like the F1-score or Precision-Recall AUC provide a balanced view. Real-world scenarios like fraud detection or medical diagnosis prioritize recall because the cost of missing a positive case (false negative) is much higher than the cost of a false alarm (false positive).
KKey Concepts to Remember
Read the scenario before looking for a memorised answer.
Find the constraint that changes the correct option.
Eliminate answers that are true in general but not in this case.
TExam Day Tips
→Watch for words such as best, first, most likely and least administrative effort.
→Review why wrong options are wrong, not only why the correct option is correct.
Key takeaway
Answer the scenario, not the keyword: identify the specific constraint before choosing the most familiar-sounding option.
Real-world example
How this comes up in practice
A startup's cloud architect reviews their monthly bill and notices costs are higher than expected for a long-running batch job. Switching from on-demand instances to Reserved Instances — or using Spot/Preemptible VMs — can reduce compute costs by up to 72 %. Questions like this test whether you understand the tradeoffs between commitment, flexibility, and cost across cloud pricing models.
What to study next
Got this wrong? Here's your next step.
Identify which exam domain this question belongs to, review the core concept, then practise similar questions from the same domain.
Fundamentals of AI and ML — This question tests Fundamentals of AI and ML — Read the scenario before looking for a memorised answer..
What is the correct answer to this question?
The correct answer is: Recall — Option D (Recall) is correct because in this highly imbalanced dataset (95% non-churners vs 5% churners), the model's 95% accuracy is misleading—it can achieve this by simply predicting the majority class (non-churner) for all samples. Recall measures the proportion of actual churners correctly identified (True Positives / (True Positives + False Negatives)), directly addressing the business need to detect churn. A high recall ensures the model captures most churners, even at the cost of some false positives.
What should I do if I get this AIF-C01 question wrong?
Identify which exam domain this question belongs to, review the core concept, then practise similar questions from the same domain.
What is the key concept behind this question?
Read the scenario before looking for a memorised answer.
About these practice questions
Courseiva creates original exam-style practice questions with explanations and wrong-answer analysis. It does not publish real exam questions, exam dumps, or protected exam content. Learn why practice questions differ from exam dumps →
Share a tip, memory trick, or ask about the reasoning behind this question. Do not post real exam questions, leaked content, braindumps, or copyrighted exam material. Comments are moderated and may be removed without notice.
This AIF-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AIF-C01 exam.
Question Discussion
Share a tip, memory trick, or ask about the reasoning behind this question. Do not post real exam questions, leaked content, braindumps, or copyrighted exam material. Comments are moderated and may be removed without notice.
Sign in to join the discussion.