AI0-001 AI Concepts and Foundations Practice Question
A data scientist is training a binary classification model to detect fraudulent transactions. The dataset has 99% legitimate transactions and 1% fraudulent. The model achieves 99% accuracy but fails to catch most fraud. Which metric should the team prioritize to evaluate model performance?
⚠ Common exam trap
CompTIA often tests the misconception that high accuracy implies good model performance, especially in imbalanced datasets, leading candidates to overlook recall as the appropriate metric for minority class detection.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Recall
Recall (sensitivity) measures the proportion of actual positive cases (fraud) correctly identified. With 99% accuracy but failing to catch most fraud, the model is biased toward the majority class (legitimate transactions), so recall is the critical metric to ensure fraud detection improves.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
F1 score
Why it's wrong here
The F1 score is incorrect because it balances precision and recall via their harmonic mean, yet in this 99:1 class imbalance, the model’s 99% accuracy is driven by correctly classifying the majority class, leaving recall for fraud near zero. The F1 score would still appear high if precision is also high, masking the failure to detect fraud. It is tempting because F1 is designed for imbalanced datasets where both false positives and false negatives matter equally; it would be correct if the cost of missing fraud and falsely flagging legitimate transactions were symmetric.
- ✗
Precision
Why it's wrong here
Precision focuses on false positives, not on catching fraud.
- ✗
Accuracy
Why it's wrong here
Accuracy is misleading on imbalanced datasets.
- ✓
Recall
Why this is correct
Recall measures the ability to catch fraudulent transactions, which is the primary goal.
About these practice questions
Courseiva writes every AI0-001 question from scratch — 754 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
Same concept, more angles
2 more ways this is tested on AI0-001
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. When evaluating a binary classification model, which two metrics are most appropriate for imbalanced datasets? (Choose two.)
medium- A.Accuracy
- B.Mean absolute error
- ✓ C.Recall
- D.R-squared
- ✓ E.Precision
Why C: Recall (Option C) is correct because it measures the proportion of actual positive cases correctly identified, which is critical in imbalanced datasets where the minority class is of primary interest. Precision (Option E) is correct because it measures the accuracy of positive predictions, helping to avoid false positives when the positive class is rare. Together, recall and precision provide a balanced view of model performance on the minority class, unlike accuracy which can be misleadingly high by simply predicting the majority class.
Variation 2. A machine learning engineer wants to evaluate a binary classifier. Which metric is MOST appropriate when the positive class is rare (e.g., 1% of total data)?
easy- A.True negative rate
- ✓ B.F1-score
- C.Mean squared error
- D.Accuracy
Why B: When the positive class is rare (e.g., 1% of total data), accuracy is misleading because a classifier that always predicts the negative class would achieve 99% accuracy. The F1-score is the harmonic mean of precision and recall, making it robust to class imbalance by focusing on the positive class performance. It is the most appropriate metric for evaluating binary classifiers on imbalanced datasets.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI0-001 practice question is part of Courseiva's free CompTIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI0-001 exam.