AI0-001 AI Implementation and Operations Practice Question
An insurance company operates an AI claims-triage model that flags suspicious claims for human review. After six months in production, the operations team observes that the model's precision has fallen steadily while recall has stayed roughly constant, and the volume of false-positive flags has grown. The data science team suspects the input data pipeline is the cause rather than the model weights. Which TWO operational checks should the team perform first to diagnose the problem? (Choose two.)
⚠ Common exam trap
The trap here is responding to a precision drop by retraining or thresholding the model, when the evidence points to upstream data corruption that must be diagnosed before any model change.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Audit the feature engineering and ETL pipeline for silent failures such as nulls, default values, or unit changes that alter feature semantics.
Stable recall with falling precision points to inputs that no longer match training conditions rather than to the model weights. Comparing production feature distributions against the training baseline detects drift or schema changes, and auditing the ETL and feature engineering stages uncovers silent failures such as nulls, defaults, or unit changes. Retraining, threshold changes, or more reviewers treat symptoms without identifying the pipeline defect.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Increase the model's decision threshold so fewer claims are flagged as suspicious.
Why it's wrong here
Raising the threshold reduces the number of flags and may lower false positives, but it does not investigate why precision declined. If the input pipeline is corrupting features, the threshold change merely hides the symptom while the underlying data problem continues to distort every prediction, including those below the new cutoff.
- ✓
Audit the feature engineering and ETL pipeline for silent failures such as nulls, default values, or unit changes that alter feature semantics.
Why this is correct
A pipeline defect that substitutes nulls, defaults, or differently scaled values changes the meaning of features without changing the model, which degrades precision while recall holds. Auditing the ETL and feature engineering stages for silent failures directly tests the team's hypothesis that the input data pipeline, not the model weights, is the root cause.
- ✓
Compare the distribution of each production input feature against the training baseline to detect upstream data drift or schema changes.
Why this is correct
Declining precision with stable recall is a classic symptom of the input distribution moving away from what the model learned. Comparing production feature distributions to the training baseline reveals covariate shift, new categories, or changed value ranges introduced upstream, which is the fastest way to confirm whether the pipeline rather than the model weights is responsible.
- ✗
Retrain the model immediately on the last six months of production data to restore precision.
Why it's wrong here
Retraining before diagnosing may bake a pipeline defect into the new model and mask the real problem, producing a model that fails again once the pipeline changes. The team explicitly suspects the input pipeline, so retraining first skips the diagnostic step and risks treating a symptom while leaving the upstream defect in place.
- ✗
Expand the human review team so more flagged claims can be examined manually while the model remains unchanged.
Why it's wrong here
Adding reviewers addresses the symptom of increased false-positive volume but provides no diagnostic information about the cause. It also raises operating cost permanently and leaves the corrupted or drifting inputs unexamined, so the model's precision would continue to erode despite the additional staffing.
About these practice questions
Courseiva writes every AI0-001 question from scratch — 962 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official CompTIA exam blueprint
This AI0-001 practice question is part of Courseiva's free CompTIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI0-001 exam.