AI0-001 Implementing AI Solutions Practice Question
During testing of an AI system that classifies support tickets into categories, the team notices the model frequently misclassifies tickets about a new product feature that was introduced after the model was trained. Which type of testing should the team prioritize to catch this issue?
⚠ Common exam trap
The AI0-001 exam often tests the distinction between testing the model's predictive behavior (regression testing) versus testing the infrastructure or data pipeline components, leading candidates to mistakenly choose unit or integration tests.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Regression testing with a test set that includes examples of the new feature
The model's misclassification of the new product feature is a classic case of data drift, where the production data distribution differs from the training data. Regression testing with a test set that includes examples of the new feature directly validates whether the model still performs correctly on this unseen category. This is the most targeted approach to catch the regression in classification accuracy caused by the new feature.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Unit tests for the data pipeline
Why it's wrong here
Unit tests verify individual code components, not the model's ability to classify new categories.
- ✓
Regression testing with a test set that includes examples of the new feature
Why this is correct
Regression testing involves re-running tests after changes; including new feature examples helps detect if the model fails on previously unseen categories.
- ✗
Integration tests for API calls
Why it's wrong here
Integration tests ensure APIs work together, but they do not evaluate model performance on new data.
- ✗
Evaluation framework for LLM output quality
Why it's wrong here
LLM evaluation frameworks are for text generation quality, not for classification accuracy on new categories.
About these practice questions
One of 754 original AI0-001 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI0-001 practice question is part of Courseiva's free CompTIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI0-001 exam.