AI0-001 AI Models and Data Engineering Practice Question
A logistics company uses a machine learning model to predict delivery times based on historical data. The model was performing well, but recently it started making inaccurate predictions, especially for routes that have experienced new traffic patterns and road closures. The data engineering team receives an alert that the model's accuracy has dropped by 15% over the last week. They suspect data drift. The team has access to the original training data and a continuous stream of new data. What is the most appropriate first step for the team to take?
⚠ Common exam trap
CompTIA often tests the misconception that the immediate response to a performance drop should be retraining or rollback, rather than first diagnosing the type of drift (data drift vs. concept drift) through distribution comparison.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Compare the distributions of key features between the training data and the recent data to quantify data drift.
The first step in diagnosing a suspected data drift is to statistically compare the distributions of key features between the training data and the recent streaming data. This quantifies whether the input data distribution has changed, which directly explains the accuracy drop. Without this analysis, any corrective action (like retraining or rollback) would be premature and could mask the root cause.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Roll back the model to the previous stable version and schedule a full audit of the data pipeline.
Why it's wrong here
Rolling back restores earlier predictions but discards the new traffic information, so the retrained model still cannot capture road closures. It is tempting as a safety measure when a bad deployment causes the drop, and would be correct if a recent model release, not genuine data drift, were the cause.
- ✓
Compare the distributions of key features between the training data and the recent data to quantify data drift.
Why this is correct
Distribution comparison directly quantifies drift by contrasting feature statistics between the original training data and recent streamed data, confirming whether new traffic patterns and closures shifted inputs. This diagnostic precedes retraining, isolating whether the 15% accuracy drop stems from input drift rather than label or concept change.
- ✗
Immediately retrain the model using the most recent data to adapt to the new patterns.
Why it's wrong here
Retraining immediately on recent data skips diagnosing whether drift is real, which features shifted, or whether labels are trustworthy, risking a worse model. It is tempting because retraining is the eventual fix, and would be correct once drift is confirmed and validated data is prepared.
- ✗
Add more features to the model to capture the new traffic patterns and road closures.
Why it's wrong here
Adding features addresses model capacity, not the distribution shift in existing inputs, and new features may be unavailable at inference. It is tempting because new traffic patterns look like missing signals, and would be correct if analysis showed the model lacked predictive inputs rather than drifting.
About these practice questions
Courseiva writes every AI0-001 question from scratch — 962 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI0-001 practice question is part of Courseiva's free CompTIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI0-001 exam.