AI-102 Practice Question: Implement natural language processing solutions
You are deploying a Conversational Language Understanding (CLU) model to production. You need to monitor the model's performance and detect when retraining is needed due to concept drift. Which metric should you monitor?
⚠ Common exam trap
Many candidates confuse operational metrics (like response time or throughput) with model performance metrics, assuming any change in usage patterns indicates drift, when in fact only a drop in prediction confidence directly reflects model uncertainty.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Average confidence scores of predictions
Average confidence scores of predictions is the correct metric because a sustained drop in confidence indicates that the model is encountering utterances that differ from its training distribution, which is a classic sign of concept drift. Monitoring confidence scores allows you to detect when the model's predictions become less certain, triggering the need for retraining with new data.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Response time for each prediction
Why it's wrong here
Prediction latency reflects infrastructure and model size, not whether utterances still map to the trained intents, so drift goes undetected. It is tempting because latency monitoring is standard for service health, and it would be correct for diagnosing performance regressions rather than semantic accuracy decay.
- ✗
Number of endpoint calls
Why it's wrong here
Endpoint call volume measures usage, not prediction quality, so degrading intent classification from concept drift stays invisible. It is tempting because traffic metrics are easy to chart and support capacity planning, and they would be correct for scaling decisions rather than detecting when utterances no longer match training data.
- ✗
Number of utterances processed per day
Why it's wrong here
Volume metric, does not indicate prediction quality.
- ✓
Average confidence scores of predictions
Why this is correct
Average confidence scores of predictions reveal declining certainty as utterances drift from training data, indicating concept drift. Monitoring this metric satisfies the requirement to detect when retraining is needed, since sustained low confidence signals the model no longer matches production input.
Go deeper
Related to this question
About these practice questions
One of 761 original AI-102 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI-102 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI-102 exam.