DA0-002 Data Analysis Practice Question
In an A/B test, the null hypothesis states that there is no difference between the conversion rates of the control and treatment groups. After collecting data, the p-value is 0.03. Using a significance level α = 0.05, what should the analyst conclude?
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Reject the null hypothesis; there is a significant difference
Since the p-value (0.03) is less than α (0.05), the null hypothesis is rejected, indicating a statistically significant difference between the groups.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Reject the null hypothesis; there is a significant difference
Why this is correct
The p-value of 0.03 falls below the 0.05 significance level, so the observed difference is unlikely under the null hypothesis. Rejecting the null is therefore the statistically correct conclusion, indicating a significant difference between control and treatment conversion rates.
- ✗
Accept the alternative hypothesis; the treatment is better
Why it's wrong here
Rejecting the null hypothesis shows a statistically significant difference, but the p-value alone cannot establish direction or that the treatment outperforms control. Concluding the treatment is better requires comparing the observed conversion rates and confidence intervals. This option is tempting because significance is often conflated with superiority, yet it overstates what the test proves.
- ✗
The test is inconclusive
Why it's wrong here
A p-value of 0.03 against α = 0.05 yields a clear reject-or-not decision, so the test is not inconclusive. Inconclusiveness would arise from insufficient sample size or a confidence interval spanning zero. This option is tempting when results sit near the threshold, but 0.03 is unambiguously below 0.05.
- ✗
Fail to reject the null hypothesis; no significant difference
Why it's wrong here
A p-value of 0.03 falls below the 0.05 significance level, so the null hypothesis is rejected. Failing to reject requires p greater than α. This conclusion is tempting when a result is close to the threshold, but 0.03 is decisively inside the rejection region, indicating a statistically significant difference between the groups.
Go deeper
Related to this question
About these practice questions
Courseiva writes every DA0-002 question from scratch — 1,004 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This DA0-002 practice question is part of Courseiva's free CompTIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the DA0-002 exam.