DA0-002 Data Analysis Practice Question
A data analyst is preparing to run an A/B test comparing two email subject lines. Which TWO of the following should the analyst define before the test begins?
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
The null and alternative hypotheses
Before A/B testing, define null and alternative hypotheses, and determine sample size needed for desired statistical power and effect size.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
The exact lift in conversion rate
Why it's wrong here
The exact lift is unknown until the test concludes; only a minimum detectable effect can be specified beforehand. It is tempting because lift is the metric of interest, yet pre-defining an expected result would bias the analysis, whereas power analysis sets the detectable effect size in advance.
- ✗
The time series decomposition
Why it's wrong here
Time series decomposition splits a series into trend, seasonal and residual components, which suits forecasting rather than a two-variant email test. It is tempting because analysts decompose historical campaign metrics, but an A/B test compares independent groups, so decomposition is irrelevant here.
- ✗
The p-value after the test
Why it's wrong here
A p-value is an output computed from collected sample data, so it cannot be fixed beforehand. It is tempting because significance thresholds feel like pre-test decisions, yet the analyst defines the alpha level and sample size in advance, not the resulting p-value itself.
- ✓
The null and alternative hypotheses
Why this is correct
Stating the null and alternative hypotheses before the A/B test begins fixes exactly what is being tested and which outcome counts as significant. This pre-registration prevents post-hoc interpretation of results and satisfies the stem's requirement to define parameters before the test starts.
- ✓
The sample size required for the desired power
Why this is correct
Calculating the required sample size from the desired statistical power, significance level and expected effect size ensures the test can detect a genuine difference. Defining this before the A/B test begins prevents underpowered results and satisfies the stem's pre-test planning requirement.
Go deeper
Related to this question
About these practice questions
Courseiva writes every DA0-002 question from scratch — 1,004 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This DA0-002 practice question is part of Courseiva's free CompTIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the DA0-002 exam.