DA0-002 Data Analysis Practice Question
A data analyst is validating a regression model that predicts monthly churn probability for 50,000 subscribers. The analyst wants to detect whether the model is overfitting before deploying it. (Choose two.)
⚠ Common exam trap
Test-takers frequently confuse actions that increase model complexity with techniques that actually diagnose overfitting.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Compare the model's error on the training set with its error on a held-out validation set
Overfitting is detected by measuring how well a model performs on data it did not train on. Holding out a validation set and comparing its error with training error reveals the generalization gap, while k-fold cross-validation repeats that comparison across multiple partitions to produce a stable estimate. Together they show whether the churn model has learned signal versus memorized noise, which is exactly what must be established before deployment.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Remove all records with missing values from the validation set only
Why it's wrong here
Deleting missing-value records from just the validation set introduces selection bias and makes the validation sample inconsistent with the training distribution. It does not measure generalization and can make the model appear better or worse for reasons unrelated to overfitting, so it is not a valid diagnostic technique here.
- ✗
Increase the number of predictor variables until training accuracy reaches 100 percent
Why it's wrong here
Adding predictors until the model perfectly fits the training data is precisely how overfitting is created, not detected. A model with 100 percent training accuracy on churn data usually has learned idiosyncratic noise, and its validation performance will degrade. This action worsens the problem the analyst is trying to diagnose.
- ✗
Apply a log transformation to the target churn probability before evaluation
Why it's wrong here
A log transformation changes the scale of the target and can affect how error is computed, but it does not by itself reveal overfitting. Overfitting is diagnosed by comparing performance on data the model has seen with performance on data it has not seen, so transforming the target without a proper holdout comparison leaves the question unanswered.
- ✓
Compare the model's error on the training set with its error on a held-out validation set
Why this is correct
A large gap between training error and validation error is the classic symptom of overfitting, because the model has memorized training noise that does not generalize. Comparing the two error values directly quantifies that gap and tells the analyst whether the model's complexity is justified, making this a core diagnostic step before deployment.
- ✓
Use k-fold cross-validation to estimate performance across multiple data splits
Why this is correct
Cross-validation partitions the data into k folds, trains on k-1 folds and validates on the remaining fold, repeating until every fold has served as validation. Averaging the results gives a more stable generalization estimate than a single split, exposing overfitting that might be hidden by one lucky or unlucky holdout partition.
Go deeper
Related to this question
About these practice questions
This DA0-002 question is part of Courseiva's 1,004-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official CompTIA exam blueprint
This DA0-002 practice question is part of Courseiva's free CompTIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the DA0-002 exam.