A/B Test Sample Size & Duration Calculator

A/B Test Sample Size & Duration Calculator

Calculate the sample size and duration needed for statistically significant A/B tests. Stop guessing and make data-driven decisions.

Test Parameters

0.5%50%
5% (high precision)50% (large effect)

Detect a change from 5% → 6.0%

Required Sample Size

Total Sample Needed
16,316
8,158 per variation × 2 variations
Estimated Duration
17 days
~3 weeks
Detecting Change
5.0% → 6.0%
+1.00 pp absolute
Test Progress17 days at 1,000 visitors/day

Day 5

25%

Day 9

50%

Day 13

75%

Day 17

100%

Statistical Power: 80%

This calculator uses 80% power (industry standard), meaning there's a 20% chance of missing a real effect. Never end a test early — at 50% sample size, the false positive rate jumps to ~30%.

Get Your Free Growth Audit

Turn these numbers into a real growth plan. We'll analyze your metrics and identify exactly where to improve profitability.

Full metrics analysisActionable recommendations100% free, no strings
Get Free Growth Audit

500+ growth audits delivered · No sales pitch

Why Sample Size Matters in A/B Testing

The most common mistake in A/B testing is ending tests too early. Without adequate sample size, you risk false positives — declaring a winner when the difference is just random noise.

Our calculator uses standard statistical power analysis (two-proportion z-test at 80% power) to determine exactly how many visitors you need before making a confident decision.

Key Concepts

Baseline Conversion Rate

Your current conversion rate before the test. Lower rates need larger samples.

Minimum Detectable Effect (MDE)

The smallest relative improvement you want to detect. A 20% MDE on a 5% baseline means detecting 5% → 6%.

Statistical Significance

Confidence that results aren't random. 95% means only 5% chance of false positive.

Statistical Power (80%)

Probability of detecting a real effect when it exists. 80% is the standard.

How Long Should You Run an A/B Test?

Run your test until both conditions are met: (1) you reach the required sample size, AND (2) you've run for at least 1-2 full business cycles (typically 7-14 days minimum).

Key Insight

A test that shows "95% significance" at only 50% of the required sample size actually has a ~30% false positive rate. Always reach full sample size before making decisions.

Tips for Faster A/B Tests

Test larger changes

Subtle tweaks need huge samples. Bold redesigns (new layouts, pricing, CTAs) reach significance faster.

Focus on high-traffic pages

Test on pages with the most visitors to reach sample size quickly.

Reduce variations

A/B tests (2 variations) need half the traffic of A/B/C tests (3 variations).

Accept 90% significance

For low-stakes tests, 90% confidence reduces required sample size by ~20%.

Need Expert Help With Your Numbers?

These calculators give you the data. Our team gives you the strategy.

Custom growth roadmapROI-focused strategyNo-commitment call
Talk to an Expert

Frequently Asked Questions

Common questions about using the A/B Test Calculator

Sample size depends on three factors: your baseline conversion rate, the minimum detectable effect (MDE) you want to measure, and your desired statistical significance level. Generally, smaller effects and higher confidence require larger sample sizes. Our calculator uses standard statistical power analysis (80% power, two-tailed test).
Run your test until you reach the required sample size AND for at least 1-2 full business cycles (typically 2-4 weeks). Never stop a test early just because results look significant — early results are often misleading due to small sample sizes and day-of-week effects.
For most websites, aim to detect a 10-20% relative improvement. For example, if your baseline conversion rate is 5%, a 20% relative MDE means you want to detect when it changes to 6% (5% × 1.20). Smaller MDEs require much larger sample sizes.
95% significance is the industry standard for most A/B tests. This means there is only a 5% chance the results are due to random variation. For high-stakes tests (pricing, major redesigns), consider 99% significance.
Ending tests early dramatically increases false positive rates. A test showing 95% significance at 50% of the required sample size actually has a ~30% false positive rate. Always reach full sample size before making decisions.

Use This With

CPM/CPC/CPA Calculator

Calculate CPM, CPC, and CPA instantly. Convert between ad metrics, compare benchmarks, and optimize your campaign performance.

Marketing ROI Calculator

Calculate the true ROI of your marketing campaigns. Compare up to 5 campaigns side-by-side to find your most profitable channels and optimize budget allocation.

Related Tools

Explore our other free calculators to optimize every aspect of your business growth

Ad Budget Calculator

Calculate your ideal advertising budget, required leads, breakeven ROAS, and expected profit. Plan profitable ad campaigns with data-driven insights.

Pricing Calculator

Calculate the right price for your services based on expenses, billable hours, and profit goals. Stop undercharging with data-driven pricing strategy.

LTV:CAC Calculator

Calculate Customer Lifetime Value (LTV), LTV:CAC ratio, and payback period. Assess the health of your unit economics and customer acquisition strategy.

Break-Even Calculator

Calculate your break-even point, revenue targets, and profit at different volumes. Visualize profitability with interactive charts and scenario comparisons.

Ecommerce Profit Calculator

Calculate your net profit per order, breakeven CPA, required ROAS, and daily profit potential. Know if your product is profitable before spending on ads.

CPM/CPC/CPA Calculator

Calculate CPM, CPC, and CPA instantly. Convert between ad metrics, compare benchmarks, and optimize your campaign performance.

ROAS Calculator

Calculate your Return on Ad Spend (ROAS), breakeven ROAS, and maximum allowable ad spend. Know if your campaigns are profitable before scaling.

Marketing ROI Calculator

Calculate the true ROI of your marketing campaigns. Compare up to 5 campaigns side-by-side to find your most profitable channels and optimize budget allocation.

UTM Link Builder

Build UTM-tagged campaign URLs instantly. Add source, medium, campaign, and content parameters. Presets for Google, Meta, and LinkedIn.

CPL Calculator

Calculate your cost per lead across channels and compare against industry benchmarks. Find your most efficient lead sources.

Meta Tag Checker

Preview how your title and meta description appear in Google results. Check character counts, pixel widths, and optimize for CTR.

Subject Line Scorer

Score email subject lines on length, power words, urgency, and spam triggers. Preview mobile appearance and get optimization tips.