AI-102 Practice Question: Implement natural language processing solutions
You are deploying a Conversational Language Understanding (CLU) model to production. You need to monitor the model's performance and detect when retraining is needed due to concept drift. Which metric should you monitor?
⚠ Common exam trap
Many candidates confuse operational metrics (like response time or throughput) with model performance metrics, assuming any change in usage patterns indicates drift, when in fact only a drop in prediction confidence directly reflects model uncertainty.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Average confidence scores of predictions
Average confidence scores of predictions is the correct metric because a sustained drop in confidence indicates that the model is encountering utterances that differ from its training distribution, which is a classic sign of concept drift. Monitoring confidence scores allows you to detect when the model's predictions become less certain, triggering the need for retraining with new data.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Response time for each prediction
Why it's wrong here
Latency metric, not indicative of accuracy.
- ✗
Number of endpoint calls
Why it's wrong here
Usage metric, not related to concept drift.
- ✗
Number of utterances processed per day
Why it's wrong here
Volume metric, does not indicate prediction quality.
- ✓
Average confidence scores of predictions
Why this is correct
Decreasing confidence suggests the model is encountering unfamiliar patterns.
Go deeper
Related to this question
About these practice questions
One of 945 original AI-102 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI-102 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI-102 exam.