DA0-002 Data Analysis Practice Question
An analyst is conducting an A/B test on a new checkout process. To calculate sample size, which THREE factors must be considered?
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Desired effect size
Statistical power, significance level (alpha), and desired effect size (minimum detectable effect) are essential for sample size calculation.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Number of control groups
Why it's wrong here
A/B testing compares one control against one treatment variant; the number of control groups is fixed at one, so it is not a sample-size input. It is tempting because multi-arm tests exist, but those are A/B/n designs. Sample size instead requires baseline rate, detectable effect and power.
- ✓
Desired effect size
Why this is correct
Desired effect size is the minimum lift worth detecting between control and variant checkout processes. Smaller effects need larger samples, so effect size is a required input when calculating sample size for the A/B test.
- ✓
Significance level (alpha)
Why this is correct
Significance level (alpha) sets the false-positive threshold, directly determining the critical value used in sample size formulas. A stricter alpha (for example 0.01 rather than 0.05) demands a larger sample to detect the same effect, so it is a required input alongside power and minimum detectable effect.
- ✓
Statistical power
Why this is correct
Statistical power, typically 80%, is the probability of detecting a real effect when one exists. It directly determines sample size: higher power requires more observations, so it must be specified when calculating sample size for the checkout A/B test.
- ✗
Population standard deviation
Why it's wrong here
Sample size for a two-proportion A/B test depends on baseline conversion rate, minimum detectable effect and significance/power levels; the population standard deviation is unknown for a binary checkout outcome. It is tempting because standard deviation drives sample size in continuous-metric tests, but proportions use variance derived from the baseline rate.
About these practice questions
Courseiva writes every DA0-002 question from scratch — 1,004 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This DA0-002 practice question is part of Courseiva's free CompTIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the DA0-002 exam.