DA0-002 Data Analysis Practice Question
A company runs an A/B test to compare a new website layout (treatment) against the current layout (control). The conversion rate for the control is 5% and for the treatment is 5.5%. The p-value is 0.06 at α=0.05. Which THREE of the following conclusions are valid?
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
There is not enough evidence to conclude that the new layout is better.
The p-value > α, so fail to reject the null; the difference is not statistically significant. However, the observed lift is 0.5% (absolute). Sample size might be insufficient; statistical power could be low.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
There is not enough evidence to conclude that the new layout is better.
Why this is correct
Correct: Fail to reject null.
- ✗
The test has sufficient power to detect the observed effect.
Why it's wrong here
p>0.05 might indicate low power; we cannot assume sufficient power.
- ✓
The observed lift of 0.5% may be due to random chance.
Why this is correct
Correct: p>0.05 suggests chance could explain the difference.
- ✗
The new layout significantly increases conversion rate.
Why it's wrong here
Not significant because p>0.05.
- ✓
A larger sample size might reveal a significant difference if one exists.
Why this is correct
Correct: Increasing sample size increases power to detect small effects.
Go deeper
Related to this question
About these practice questions
This DA0-002 question is part of Courseiva's 986-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This DA0-002 practice question is part of Courseiva's free CompTIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the DA0-002 exam.