AI0-001 Machine Learning and Deep Learning Practice Question
A financial firm trained a gradient boosting model on two years of loan data. It reported strong AUC during development, but after six months in production, approval rates for a newly launched loan product diverge sharply from expectations. The data science lead suspects the model is stale. Which approach best addresses this deployment issue?
⚠ Common exam trap
The trap here is treating a production performance drop as a tuning problem, when the real cause is that the training distribution no longer matches the live applicant population.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Monitor prediction distributions and retrain on recent labeled data when drift is detected
The symptoms describe drift: a new product changes the applicant distribution and recent economic conditions change the relationship between features and default. The durable fix is to monitor for drift and retrain on recent labeled data, then revalidate before deployment. Changing hyperparameters, adding boosting rounds, or swapping metrics leaves the model trained on outdated patterns and cannot restore alignment with current production behavior.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Retrain the model on the original two-year dataset with a lower learning rate
Why it's wrong here
Lowering the learning rate changes optimization dynamics but does not incorporate the new product's data, so the model remains blind to the changed applicant population and economic conditions. Retraining on the same stale window reproduces the same stale decision boundaries. This action addresses training stability rather than the distribution shift that is causing the production divergence, leaving the core problem unresolved.
- ✗
Switch the evaluation metric from AUC to accuracy without changing the data
Why it's wrong here
Changing the reported metric does not alter the model's parameters or its exposure to stale data, so approval behavior in production is unchanged. It only changes how performance is summarized and may obscure the drift by using a threshold-dependent metric. The divergence stems from distribution shift, not from the choice of evaluation statistic, so metric substitution fails to remediate the underlying staleness.
- ✗
Increase the number of boosting rounds to improve fit on the existing data
Why it's wrong here
Adding boosting rounds fits the historical data more closely but cannot teach the model about applicants and outcomes that did not exist in the original window. It risks overfitting the stale distribution and may make predictions worse for the new product. More capacity is not a substitute for current data, so this action misdiagnoses a drift problem as an underfitting problem.
- ✓
Monitor prediction distributions and retrain on recent labeled data when drift is detected
Why this is correct
This combines drift detection with periodic retraining on fresh, labeled outcomes, which is the standard remedy for data and concept drift. Monitoring feature and prediction distributions catches when the new loan product shifts the input space, and retraining on recent labels updates the model's learned relationships. It directly addresses staleness while preserving the ability to validate the refreshed model before redeployment in a regulated lending context.
About these practice questions
One of 962 original AI0-001 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official CompTIA exam blueprint
This AI0-001 practice question is part of Courseiva's free CompTIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI0-001 exam.