Choosing Evaluation Metrics for Imbalanced Data: F1, F2, Precision, Recall
An AI team is deploying a predictive maintenance model for industrial equipment. The model predicts failure within a 30-day window. The cost of a false positive is 10% of the cost of a false negative. Which evaluation metric should the team prioritize?
Quick Answer
The answer is the F2 score (beta=2), which prioritizes recall over precision. This is correct because when false negatives are ten times costlier than false positives, the evaluation metric must emphasize minimizing missed failures, and the F-beta formula with beta=2 weights recall four times higher than precision, directly aligning with the asymmetric cost structure. On the CompTIA AI+ AI0-001 exam, this scenario tests your understanding of cost-sensitive evaluation metrics for imbalanced data, where the common trap is choosing F1 (beta=1) or AUC-ROC, which ignore cost ratios. A reliable memory tip: remember that beta > 1 means you care more about catching failures (recall), so for a costlier false negative, "beta bigger, recall bigger."
⚠ Common exam trap
It's easy for candidates to default to F1 score as a 'balanced' metric without considering the asymmetric cost structure, or they may incorrectly think AUC-ROC captures cost-sensitive performance.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
F2 score (beta=2) to prioritize recall over precision.
The F2 score (beta=2) weights recall four times more than precision, which is appropriate because a false negative (missing a failure) costs 10 times more than a false positive (unnecessary maintenance). Prioritizing recall ensures the model captures as many true failures as possible, minimizing the higher-cost error type.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
F2 score (beta=2) to prioritize recall over precision.
Why this is correct
F2 score puts more weight on recall, aligning with the higher cost of false negatives.
- ✗
Area under the ROC curve (AUC-ROC) to measure overall discrimination.
Why it's wrong here
AUC-ROC does not directly reflect the cost-sensitive decision threshold.
- ✗
F1 score to balance precision and recall equally.
Why it's wrong here
F1 gives equal weight, but false negatives are costlier than false positives.
- ✗
Precision to minimize false positives.
Why it's wrong here
False positives are less costly than false negatives, so recall is more important.
About these practice questions
One of 754 original AI0-001 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
Same concept, more angles
2 more ways this is tested on AI0-001
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. An AI system is deployed to detect fraudulent transactions. The system flags 5% of transactions as fraudulent, but the actual fraud rate is 0.1%. The business sees many false positives and wants to reduce them without significantly increasing false negatives. Which metric should be prioritized for optimization?
hard- A.Recall
- ✓ B.F1 score
- C.Accuracy
- D.Precision
Why B: The F1 score balances precision and recall, making it ideal when false positives are costly but false negatives must not increase significantly. Optimizing precision alone would reduce false positives but could increase false negatives, while recall alone would not address the false positive problem. The F1 score ensures both metrics are jointly optimized, aligning with the business requirement.
Variation 2. An AI model for detecting fraudulent transactions has high precision but low recall. Which business impact is most likely?
medium- A.The model has no impact on fraud detection
- B.The model detects all fraudulent transactions
- ✓ C.Many fraudulent transactions go undetected
- D.Many legitimate transactions are flagged as fraud
Why C: High precision means that when the model flags a transaction as fraudulent, it is very likely correct. However, low recall indicates that the model misses a significant proportion of actual fraudulent transactions. Therefore, the most likely business impact is that many fraudulent transactions go undetected, leading to financial losses.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI0-001 practice question is part of Courseiva's free CompTIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI0-001 exam.