DA0-002 Data Analysis Practice Question
An analyst is conducting an A/B test on a new website layout. Which TWO of the following must be defined before the test begins?
⚠ Common exam trap
DA0-002 often tests the confusion between pre-test design parameters (alpha, sample size, MDE) and post-test outputs (p-value, confidence interval, observed conversion rates), so candidates must recognize which values can only exist after data collection.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
The significance threshold (alpha)
Option D is correct because the significance threshold (alpha), typically set at 0.05, must be fixed in advance to determine the probability of a Type I error and to provide the criterion against which the resulting p-value will be judged. Option E is correct because the required sample size must be calculated beforehand, based on the desired statistical power (commonly 0.80), the baseline conversion rate, and the minimum detectable effect, to ensure the test can reliably detect a real difference. Options A, B, and C are not pre-test definitions: the final conversion rates and the actual p-value are outcomes observed only after the test concludes, and the confidence interval for the lift is likewise a post-hoc computed estimate rather than an input specified before the test begins.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
The final conversion rates for each group
Why it's wrong here
Final conversion rates are measured outcomes that only exist once the test has run and data is collected. Pre-test definition covers the primary metric, hypothesis, sample size and duration. Conversion rates would be the correct focus when analysing results and deciding whether to ship the new layout.
- ✗
The actual p-value from the test
Why it's wrong here
The p-value is calculated from the observed data at analysis time; it cannot be specified in advance. Pre-test planning fixes the significance threshold, metric and sample size instead. A p-value would be the right quantity to interpret when determining whether the observed difference is statistically significant.
- ✗
The confidence interval for the lift
Why it's wrong here
The confidence interval is an output computed from collected sample data, not an input fixed beforehand. Before testing you specify the hypothesis, primary metric, sample size and significance level. A confidence interval would be the right deliverable when reporting the estimated range of the observed lift after analysis.
- ✓
The significance threshold (alpha)
Why this is correct
Fixing alpha before the test begins controls the false-positive rate and prevents p-hacking after results arrive. The stem demands pre-defined parameters; without a declared significance threshold, the analyst cannot objectively judge whether the observed difference between layouts is statistically meaningful.
- ✓
The sample size required for adequate statistical power
Why this is correct
Statistical power depends on the minimum sample size per variant, which must be calculated from the baseline conversion rate, minimum detectable effect and significance level. Fixing it before launch satisfies the validity requirement, preventing peeking and underpowered results.
Go deeper
Related to this question
About these practice questions
This DA0-002 question is part of Courseiva's 1,004-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official CompTIA exam blueprint
This DA0-002 practice question is part of Courseiva's free CompTIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the DA0-002 exam.