AI0-001 Machine Learning and Deep Learning Practice Question
A team is deploying a sentiment classifier and notices that the model outputs probabilities such as 0.83 for the positive class, but the actual positive rate among examples scored near 0.83 is only about 0.55. Stakeholders need the scores to reflect true likelihoods. Which action should the team take?
⚠ Common exam trap
The trap here is responding to a miscalibration complaint by changing the decision threshold, which alters classifications rather than the meaning of the probability scores.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Apply a calibration method such as Platt scaling or isotonic regression to the model's output scores.
The scenario describes a reliability problem: scores near 0.83 correspond to an actual positive rate near 0.55, so the probabilities are overconfident. Calibration techniques such as Platt scaling and isotonic regression learn a mapping from raw scores to empirical frequencies using a held-out set, producing probabilities that better reflect true likelihoods without changing the model's ranking ability.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Replace the classifier with a k-nearest neighbors model, which produces inherently calibrated probabilities.
Why it's wrong here
K-nearest neighbors does not produce inherently calibrated probabilities; its class-frequency estimates depend heavily on k and the local density of the data, and can be poorly calibrated in high-dimensional feature spaces. Swapping the entire model to fix a calibration symptom is disproportionate and may degrade accuracy. Calibration methods are the appropriate, targeted fix.
- ✗
Raise the decision threshold from 0.5 to 0.83 so only highly confident examples are labeled positive.
Why it's wrong here
Moving the threshold changes the precision-recall tradeoff but does nothing to make the scores themselves meaningful probabilities. Stakeholders specifically need the numeric output to reflect true likelihood, and thresholding only affects the hard-label decision. The underlying miscalibration of the 0.83 score would remain unchanged.
- ✗
Retrain the classifier with a lower learning rate and more epochs to improve probability estimates.
Why it's wrong here
Learning rate and epoch count affect optimization and convergence, not the calibration of the output distribution. A perfectly converged model can still be miscalibrated because the loss function optimizes discrimination rather than probability accuracy. Changing training hyperparameters may alter scores slightly but will not systematically align predicted probabilities with observed frequencies.
- ✓
Apply a calibration method such as Platt scaling or isotonic regression to the model's output scores.
Why this is correct
The stated problem is miscalibration: predicted probabilities do not match observed frequencies. Platt scaling fits a logistic transform on held-out scores, and isotonic regression fits a monotonic step function; both map raw scores to calibrated probabilities. This directly addresses the mismatch between 0.83 predicted and roughly 0.55 observed, without changing the model's ranking of examples.
About these practice questions
One of 962 original AI0-001 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official CompTIA exam blueprint
This AI0-001 practice question is part of Courseiva's free CompTIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI0-001 exam.