AI0-001 Machine Learning and Deep Learning Practice Question
A machine learning engineer trains a decision tree to predict customer churn. The tree achieves 99 percent accuracy on the training set but only 68 percent on a held-out test set. The engineer wants to reduce this gap. Which single action is most likely to improve test performance?
⚠ Common exam trap
The trap here is equating higher training accuracy with a better model, when a near-perfect training score alongside poor test performance is the classic signature of overfitting.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Limit the tree depth and require a minimum number of samples per leaf
The large train-test gap signals high variance, meaning the unconstrained decision tree memorized training noise. Pre-pruning through maximum depth and minimum samples per leaf limits how finely the tree can split, reducing variance and improving held-out accuracy without discarding the model or leaking test data.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Limit the tree depth and require a minimum number of samples per leaf
Why this is correct
Constraining depth and requiring a minimum samples per leaf restricts the tree from creating tiny, noise-driven splits. This regularization reduces variance and typically narrows the train-test gap. For a tree already at 99 percent training accuracy, these hyperparameters directly address the overfitting that causes the 31-point drop on held-out data.
- ✗
Train the tree on the test set as well so it learns the held-out distribution
Why it's wrong here
Training on the test set violates the fundamental separation between training and evaluation data, producing optimistically biased metrics that no longer estimate generalization. It does not fix overfitting; it simply contaminates the only unbiased estimate of how the model will perform on unseen churn customers.
- ✗
Increase the maximum depth of the tree so it can fit more training examples
Why it's wrong here
Increasing maximum depth lets the tree carve finer partitions and memorize noise in the training data, which widens the gap between training and test accuracy rather than closing it. The model is already overfitting at high training accuracy, so granting it more capacity worsens generalization instead of improving held-out performance.
- ✗
Add more features derived from the training set to give the tree more signal
Why it's wrong here
Adding engineered features expands the hypothesis space and gives an already overfit tree more opportunities to memorize training-specific quirks. Without regularization or validation, additional features usually increase variance, so the training accuracy stays high while test accuracy stagnates or declines further.
About these practice questions
One of 962 original AI0-001 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official CompTIA exam blueprint
This AI0-001 practice question is part of Courseiva's free CompTIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI0-001 exam.