AI0-001 AI Implementation and Operations Practice Question
A data scientist is monitoring a deployed image classification model. Which TWO actions are best practices for detecting model drift? (Choose 2.)
⚠ Common exam trap
CompTIA often tests the distinction between detection and remediation actions, so candidates mistakenly choose retraining (Option A) as a detection method when it is actually a corrective action.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Use a holdout test set to periodically evaluate model accuracy.
Option C is correct because periodically scoring the model against a fixed holdout test set gives a direct, repeatable measurement of accuracy degradation, which is the core signal of model drift. Option E is correct because tracking the distribution of input features over time (e.g., via histograms or statistical tests like KL divergence or PSI) detects data drift, which typically precedes and causes model drift. Option A is not a detection practice but a remediation action, and retraining blindly on a schedule does not identify drift. Option B is a modeling change that may affect generalization but does nothing to detect drift in a deployed model. Option D is weaker than C and E because average prediction confidence can shift for reasons unrelated to drift and is not a reliable standalone drift indicator.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Schedule automatic weekly retraining of the model.
Why it's wrong here
Scheduled retraining alters the model rather than detecting drift; it can mask degradation and wastes compute when data is stable. It is tempting because retraining is the eventual remedy once drift is confirmed, and it would be correct as a remediation step after monitoring has flagged a genuine distributional shift.
- ✗
Increase the model's complexity to improve generalization.
Why it's wrong here
Adding complexity raises capacity and overfitting risk without comparing live inputs against training distributions, so drift remains undetected. It is tempting because underfitting looks like degradation, and it would be correct when validation curves show the model cannot capture the underlying pattern, not when detecting drift.
- ✓
Use a holdout test set to periodically evaluate model accuracy.
Why this is correct
A holdout test set provides labelled ground truth, letting the team measure accuracy decay against known-correct outputs over time. This detects concept drift that unlabelled input monitoring alone cannot confirm, satisfying the requirement to identify genuine performance degradation.
- ✗
Monitor the average prediction confidence of the model.
Why it's wrong here
Confidence is a model-internal score that can remain high under shifted inputs, so it cannot reliably evidence drift on its own. It is tempting because it needs no labels and is cheap to log, and it would be correct as a supplementary signal alongside input-distribution monitoring, not as a standalone drift detector.
- ✓
Track the distribution of input data over time.
Why this is correct
Tracking input distribution over time exposes covariate shift, where production data diverges from training data. This detects drift before labels arrive, complementing accuracy checks and satisfying the requirement to catch changes in the data feeding the classifier.
About these practice questions
This AI0-001 question is part of Courseiva's 962-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI0-001 practice question is part of Courseiva's free CompTIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI0-001 exam.