AI0-001 AI Implementation and Operations Practice Question
An ML operations team needs to monitor a deployed model's performance. Which TWO metrics are most useful for detecting concept drift in a regression model? (Choose two.)
⚠ Common exam trap
CompTIA often tests the distinction between covariate drift and concept drift, trapping candidates who think monitoring input features is sufficient for detecting all types of model degradation.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Distribution of residuals between predictions and actuals
Option B is correct because the distribution of residuals between predictions and actuals directly reveals concept drift: if the relationship between inputs and target changes, the residual distribution will shift (e.g., become biased or wider) even when input features look unchanged. Option E is correct because tracking MAE over a sliding time window is a standard regression performance monitor; a sustained increase in MAE relative to a baseline indicates that the model's learned mapping no longer matches the current data-generating process, which is the practical signature of concept drift. Option A is not the best choice here because input feature distribution shifts indicate data drift (covariate shift), not concept drift, and inputs can drift without the input-target relationship changing. Option C is wrong because classification accuracy applies to classification models, not regression. Option D is wrong because inference latency is an operational/system metric and says nothing about changes in the input-target relationship.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Distribution of input features
Why it's wrong here
Input feature distributions detect data drift, where the covariate distribution shifts while the feature-label relationship stays fixed; concept drift specifically requires the mapping itself to change. Feature monitoring is correct for catching upstream data pipeline or population shifts before predictions degrade.
- ✓
Distribution of residuals between predictions and actuals
Why this is correct
Residual distributions reveal concept drift because changing feature-label relationships alter the error pattern, not merely its magnitude. Tracking how residuals shift over time exposes degradation that aggregate accuracy scores can mask, making it a direct signal of drift in regression models.
- ✗
Classification accuracy
Why it's wrong here
Classification accuracy applies to categorical targets, so it cannot be computed for a regression model predicting continuous values. Concept drift in regression is tracked with error metrics such as MAE or RMSE against delayed ground truth; accuracy would be correct for a classifier detecting label drift.
- ✗
Model inference latency
Why it's wrong here
Inference latency measures serving infrastructure performance, not the statistical relationship between features and labels, so it cannot reveal concept drift. Latency monitoring is correct for operational concerns such as throughput, capacity planning, and detecting degraded or overloaded endpoints.
- ✓
Mean absolute error (MAE) over a sliding time window
Why this is correct
MAE computed over a sliding window tracks performance degradation as it emerges, since concept drift typically manifests as gradually rising prediction error. Comparing recent MAE against a baseline window flags drift promptly, satisfying the need for ongoing regression monitoring.
About these practice questions
This AI0-001 question is part of Courseiva's 962-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI0-001 practice question is part of Courseiva's free CompTIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI0-001 exam.