Classification Threshold Tuning for Recall and Precision
A retail company deploys a machine learning model to predict customer churn. The model outputs a probability between 0 and 1, and churn is predicted if probability > 0.5. After deployment, the model has a high false positive rate (many non-churning customers labeled as churn), which leads to unnecessary retention offers and increased costs. The data science team confirms the model was trained on historical data with a balanced class distribution. The business team wants to reduce false positives while maintaining a reasonable true positive rate. However, they cannot retrain the model because the original training data is no longer available. What is the best course of action to reduce false positives?
⚠ Common exam trap
CompTIA often tests the misconception that retraining or collecting more data is the only way to fix model performance issues, when in fact threshold tuning is a valid post-deployment technique that does not require retraining.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Increase the decision threshold to a higher value, such as 0.7.
Increasing the decision threshold to a higher value, such as 0.7, reduces false positives because the model will only predict churn when it is more confident. Since the model cannot be retrained, adjusting the threshold is the only way to trade off between precision and recall without modifying the model itself.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Retrain the model using only the most recent three months of data.
Why it's wrong here
Retraining is impossible here because the original training data no longer exists, and three months of recent data would also skew the balanced class distribution. Retraining is the right remedy when representative historical data remains available and the model itself must be rebuilt.
- ✓
Increase the decision threshold to a higher value, such as 0.7.
Why this is correct
Raising the threshold above 0.5 requires stronger churn evidence before labelling a customer positive, directly reducing false positives. It needs no retraining, so it works despite the unavailable training data, though some true positives are lost.
- ✗
Collect new labeled data and perform transfer learning from the original model.
Why it's wrong here
Transfer learning still requires retraining the classifier head on new labelled data, which the business has ruled out and which may not reproduce the original feature distribution. It is tempting because transfer learning reuses a pretrained model, and it would be correct if labelled churn examples were obtainable and retraining were permitted.
- ✗
Decrease the decision threshold to a lower value, such as 0.3.
Why it's wrong here
Lowering the threshold to 0.3 flags more customers as churners, increasing false positives rather than reducing them. It is tempting because threshold tuning is the standard no-retraining lever, but the correct direction is upward, toward 0.7, to demand stronger evidence before predicting churn.
About these practice questions
Courseiva writes every AI0-001 question from scratch — 962 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI0-001 practice question is part of Courseiva's free CompTIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI0-001 exam.