DA0-002 Data Analysis Practice Question
An analyst is conducting an A/B test to compare two website designs. The null hypothesis is that there is no difference in conversion rates. The p-value obtained is 0.03, and the significance threshold is 0.05. What should the analyst conclude?
⚠ Common exam trap
The trap is interpreting a significant p-value as proof that the alternative hypothesis is true in a specific direction (e.g., new design is better), when it only indicates a difference.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Reject the null hypothesis; there is a significant difference.
In hypothesis testing, if the p-value (0.03) is less than the significance level (0.05), you reject the null hypothesis. This indicates that there is statistically significant evidence of a difference in conversion rates between the two designs. The correct conclusion is to reject the null hypothesis.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Reject the null hypothesis; there is a significant difference.
Why this is correct
A p-value of 0.03 falls below the 0.05 significance threshold, so the null hypothesis of equal conversion rates is rejected. The result is statistically significant, indicating the observed difference between the two website designs is unlikely to arise from chance alone.
- ✗
Accept the alternative hypothesis that the new design is better.
Why it's wrong here
Rejecting the null shows a statistically significant difference, but the p-value alone cannot establish direction, so claiming the new design is better overstates the evidence. This conclusion would be valid only if the test were one-tailed and the observed effect favoured the new design.
- ✗
The test is inconclusive; need a larger sample size.
Why it's wrong here
A p-value of 0.03 against a 0.05 threshold yields a decisive result, so declaring the test inconclusive misreads the evidence. Inconclusiveness applies when p sits near the threshold or confidence intervals span zero, prompting a larger sample to detect a smaller effect.
- ✗
Fail to reject the null hypothesis; there is no significant difference.
Why it's wrong here
With p = 0.03 below the 0.05 threshold, the null hypothesis is rejected, so failing to reject contradicts the observed result. This conclusion would be right only if p exceeded 0.05, indicating insufficient evidence against no difference between the designs.
Go deeper
Related to this question
About these practice questions
Courseiva writes every DA0-002 question from scratch — 1,004 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official CompTIA exam blueprint
This DA0-002 practice question is part of Courseiva's free CompTIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the DA0-002 exam.