AI-102 Practice Question: Implement natural language processing solutions
You are developing a custom text classification model using Azure AI Language. The model must classify customer support tickets into 15 categories. You have 10,000 labeled examples. After training, the model shows 95% accuracy on the test set but only 60% on a small sample of new tickets. What is the most likely cause?
⚠ Common exam trap
A common mix-up: candidates confuse overfitting with data leakage or non-representative data, but the key clue is the large gap between high test accuracy and low real-world accuracy, which is the hallmark of overfitting in Azure AI Language models.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
The model is overfitting to the training data.
The model's 95% accuracy on the test set versus 60% on new tickets is a classic symptom of overfitting. In Azure AI Language custom text classification, overfitting occurs when the model learns noise and idiosyncrasies of the training data rather than generalizable patterns, causing poor performance on unseen data. The high accuracy on the test set but sharp drop on new tickets indicates the model memorized the training distribution and fails to generalize.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
The model's confidence threshold is set too low.
Why it's wrong here
Confidence threshold affects prediction rejection, not accuracy.
- ✗
The training data is not representative of the new tickets.
Why it's wrong here
While possible, the high test accuracy suggests the model learned the training data well, not that the training data was unrepresentative.
- ✗
There is data leakage between the training and test sets.
Why it's wrong here
Data leakage would inflate test accuracy, but the test accuracy is already high and the issue is with new data.
- ✓
The model is overfitting to the training data.
Why this is correct
Overfitting leads to high training accuracy but poor generalization.
Go deeper
Related to this question
About these practice questions
This AI-102 question is part of Courseiva's 945-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI-102 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI-102 exam.