A/B Test Sample Size Calculator

Baseline conversion, %

Minimum uplift, %

Confidence level, %

Statistical power, %

Visitors a day

Result

Per variant53,227
Visitors in total106,454
Days to run107
Expected conversion B, %3.3%
Confidence, %95%
Power, %80%

Worked out before the test, not after

How many visitors each variant needs to detect a given uplift, and how many days that will take.

Sample size is decided in advance, or the test becomes guesswork. Three inputs settle it: current conversion rate, the smallest uplift you want to detect, and how confident the conclusion has to be.

The surprise is how expensive small uplifts are. Detecting a ten percent lift on a three percent base needs about seventy thousand visitors per variant; detecting a two percent lift needs over a million.

What this calculator shows

Visitors needed per variant and in total
How many days that takes at your traffic
The variant B conversion rate the test would reliably distinguish

What to keep in mind

Uses the standard two-proportion sample size formula for a two-tailed test.
Eighty percent power is the accepted minimum: at that setting the test misses a real effect one time in five.

FAQs

What is the minimum detectable effect?

The smallest uplift the test can reliably tell from noise. The smaller it is, the more data you need: halving the effect quadruples the sample.

What is statistical power?

The probability of spotting an effect that is genuinely there. At eighty percent power, a real improvement is missed one time in five. Raising it to ninety costs roughly a third more sample.

Can I speed a test up?

Only by adding traffic, accepting a larger detectable effect, or raising the base conversion rate. Cutting the sample short for speed makes the conclusion worthless.

What if I do not have the traffic?

Test radical changes rather than small ones, pick a metric higher in the funnel where conversion is greater, or drop A/B testing in favour of qualitative research.

Worked example

A landing page test

Input: 3 percent base, 10 percent uplift, 95 percent confidence, 80 percent power, 1,000 visitors a day

Output: About 69,000 per variant, 138,000 in total — 138 days

Note: Nearly five months for one test. That is the honest answer, and it explains why low-traffic sites should test bold hypotheses with double-digit expected uplift rather than button shades.

Useful links

All tools in Marketing

A/B Test Sample Size Calculator

Result

Related tools

A/B Test Significance Calculator

Conversion Rate Calculator

Confidence Interval Calculator

Useful links