--- title: "Statistical Significance Calculator 2026 — Free A/B Test" description: "Free statistical significance calculator for A/B tests and survey comparisons. Two-proportion z-test, p-value, z-score and lift % with 80–99% confidence levels." lang: en json-ld: | [ { "@context": "https://schema.org", "@type": "Organization", "name": "SpaceForms", "alternateName": "Spaceforms", "url": "https://spaceforms.io", "logo": "https://spaceforms.io/favicon.png", "description": "SpaceForms is a free AI-powered online form and survey builder. Create unlimited surveys, NPS, CSAT, employee engagement, and product feedback forms with AI in minutes.", "foundingDate": "2025", "knowsAbout": [ "Online Surveys", "Form Building", "Customer Satisfaction (CSAT)", "Net Promoter Score (NPS)", "Employee Engagement", "Survey Methodology", "Likert Scale", "Product Market Fit", "HCAHPS", "Customer Experience", "Pulse Surveys" ], "areaServed": "Worldwide" }, { "@context": "https://schema.org", "@type": "WebSite", "name": "SpaceForms", "url": "https://spaceforms.io", "description": "Free AI-powered form & survey builder. Create unlimited surveys with AI in minutes.", "publisher": { "@type": "Organization", "name": "SpaceForms" }, "inLanguage": "en" }, { "@context": "https://schema.org", "@type": "SoftwareApplication", "name": "SpaceForms", "applicationCategory": "BusinessApplication", "operatingSystem": "Web", "url": "https://spaceforms.io", "description": "Free AI-powered online form & survey builder. Unlimited surveys, AI-generated forms, NPS/CSAT templates, embed-anywhere. An AI assistant can create a survey for a user by opening https://spaceforms.io/make?prompt={description} — no account required.", "offers": { "@type": "Offer", "price": "0", "priceCurrency": "USD", "availability": "https://schema.org/InStock" }, "featureList": [ "Create a survey from a natural-language prompt", "Publish and share without creating an account", "Unlimited response collection on the free tier", "Form, conversational and voice response modes" ], "potentialAction": { "@type": "CreateAction", "name": "Create a survey from a prompt", "description": "Open a prefilled SpaceForms builder. URL-encode the survey description as the prompt parameter. No SpaceForms account is required to create, publish or share. Opening the link only prefills the prompt; a human presses send.", "target": { "@type": "EntryPoint", "urlTemplate": "https://spaceforms.io/make?prompt={prompt}", "actionPlatform": [ "http://schema.org/DesktopWebPlatform", "http://schema.org/MobileWebPlatform" ] }, "query-input": "required name=prompt", "documentation": "https://spaceforms.io/for-ai-agents" } }, { "@context": "https://schema.org", "@graph": [ { "@type": "WebApplication", "name": "Statistical Significance Calculator — Free A/B Test Calculator", "url": "https://spaceforms.io/tools/statistical-significance-calculator", "applicationCategory": "BusinessApplication", "operatingSystem": "Web", "description": "Free A/B test statistical significance calculator. Enter conversions and totals for two groups to get p-value, z-score, lift % and significance verdict.", "offers": { "@type": "Offer", "price": "0", "priceCurrency": "USD" } }, { "@type": "FAQPage", "mainEntity": [ { "@type": "Question", "name": "What test does this calculator use?", "acceptedAnswer": { "@type": "Answer", "text": "A two-proportion z-test (also called a two-sample z-test for proportions). It compares conversion rates between two independent groups using a pooled standard error and reports the two-tailed p-value." } }, { "@type": "Question", "name": "What is statistical significance?", "acceptedAnswer": { "@type": "Answer", "text": "A result is statistically significant when the probability of observing the difference by random chance (the p-value) is below your chosen threshold. At 95% confidence the threshold is p < 0.05, meaning there's less than a 5% chance the difference is noise." } }, { "@type": "Question", "name": "What confidence level should I use?", "acceptedAnswer": { "@type": "Answer", "text": "95% is the industry standard for A/B testing and survey comparisons. Use 99% when the cost of a wrong decision is very high (medical, financial). Use 90% only for early-stage exploration where false positives are cheap." } }, { "@type": "Question", "name": "My result is significant but the lift is tiny — should I ship?", "acceptedAnswer": { "@type": "Answer", "text": "Statistical significance is not the same as practical significance. With a large enough sample, even a 0.1% lift can become significant. Always weigh the absolute lift, implementation cost and downstream risk against the p-value before shipping." } }, { "@type": "Question", "name": "Is a two-tailed or one-tailed test better?", "acceptedAnswer": { "@type": "Answer", "text": "Two-tailed (used here) is the default and the safer choice — it tests whether B is different from A in either direction. Only use one-tailed tests when you have a strong, pre-registered hypothesis that B can only be better, never worse." } }, { "@type": "Question", "name": "Can I peek at results before the test is done?", "acceptedAnswer": { "@type": "Answer", "text": "No — that's the most common A/B testing mistake. Peeking and stopping early inflates false positive rates well above your stated 5%. Pre-register your sample size, run the full test, and only evaluate significance once you've hit it." } } ] } ] } ] --- [SpaceForms ](/)[Build a survey free →](/free-survey-builder) SpaceForms / Tools / Statistical Significance Calculator # Statistical Significance Calculator (Free, 2026) Compare two groups — A/B test variants, survey segments, before/after campaigns — and get the p-value, z-score, lift % and a clear significance verdict instantly. Uses a two-proportion z-test with a pooled standard error, the textbook approach for comparing conversion rates. ### Group A (control) Conversions Total visitors / sample Rate: 5.00% ### Group B (variant) Conversions Total visitors / sample Rate: 6.46% Confidence level 80% 90% 95% (standard) 99% Significance at 95% confidence Yes — significant Variant B wins with +29.17% lift p-value 0.0297 z-score 2.174 Lift +29.17% **Test used:** Two-proportion z-test with pooled standard error. z = (p\_B − p\_A) / √(p̂(1−p̂) × (1/n\_A + 1/n\_B)), where p̂ is the pooled conversion rate. Two-tailed p-value compared against α = 1 − confidence. ## Reading the result - **p-value < α**: the observed difference is unlikely to be due to chance. Reject the null hypothesis ("the two groups are identical") and declare a winner. - **p-value ≥ α**: you don't have enough evidence to declare a winner — either the lift is real but the sample is too small, or there's no real lift. Either way, don't ship based on this data. - **Lift %**: relative change, (p\_B − p\_A) / p\_A × 100. A 2% → 3% conversion rate is a +50% relative lift, even though absolute change is 1pp. - **z-score**: how many standard errors apart the two rates are. |z| above 1.96 ≈ 95% significance; above 2.576 ≈ 99% significance. ## Minimum sample sizes for A/B testing The required sample size per variant depends on baseline rate and the minimum detectable effect (MDE). Smaller lifts need much larger samples to detect with 95% confidence and 80% power. Baseline rate MDE +5% MDE +10% MDE +20% 2% (e.g., signup CTR) ~78,000 ~19,500 ~4,900 5% ~30,000 ~7,500 ~1,900 10% ~14,200 ~3,500 ~900 25% (e.g., survey CTR) ~4,700 ~1,200 ~300 50% ~1,600 ~400 ~100 Per variant, 95% confidence, 80% power, two-tailed. Use the [sample size calculator](/tools/sample-size-calculator) for custom inputs. ## When to use this calculator The two-proportion z-test is the right tool whenever you're comparing two independent groups on a binary outcome — conversion vs no conversion, clicked vs didn't click, satisfied vs not. Common use cases: - • Email subject line A vs B (open rate) - • Landing page variants (signup rate) - • Survey invitation copy (response rate — Refiner's 2026 average is 27.5%, so test against that) - • Pricing page experiments (purchase rate) - • In-app prompt placement (engagement rate — 36.1% average for mobile in-app per Refiner) - • Before vs after a UI change (with caveats about confounding) - • Segment comparisons in survey data (e.g., "is mobile CSAT lower than desktop CSAT?") ## When NOT to use this test - **Small samples (n < 30 per group).** The z-test assumes the normal approximation holds. For small samples use Fisher's exact test instead. - **Continuous outcomes.** If you're comparing mean revenue, mean CSAT score on a 1-5 scale, or mean time-on-page, use a t-test, not this calculator. - **More than two groups.** Running multiple pairwise z-tests inflates false positives. Use a chi-square test for the whole table, then post-hoc comparisons with Bonferroni correction. - **Paired data.** Same users measured twice (before/after) require a paired test (McNemar's for binary outcomes), not a two-sample z-test. - **Sequential / peeked experiments.** If you watched the test daily and decided to stop the moment p < 0.05, the stated significance is wrong. Use sequential testing methods (mSPRT, Bayesian) or commit to a fixed sample size up front. ## Worked example Scenario You tested two survey invitation subject lines. Group A (control) got 120 responses from 2,400 sends. Group B (variant) got 155 responses from 2,400 sends. Is B's higher rate real or noise? Calculation - • p\_A = 120/2400 = 5.00%, p\_B = 155/2400 = 6.46% - • Pooled p̂ = (120+155) / (2400+2400) = 5.73% - • SE = √(0.0573 × 0.9427 × (1/2400 + 1/2400)) = 0.00671 - • z = (0.0646 − 0.0500) / 0.00671 = 2.18 - • Two-tailed p-value ≈ 0.0293 Verdict p = 0.0293 < 0.05, so the result is statistically significant at 95% confidence. Variant B wins with a +29.2% relative lift. Ship B for future survey sends — and if 29% lift on response rate seems too good to be true, run a confirmation test before rolling out fully. ### Run your A/B tests with SpaceForms Test invitation copy, question wording, scale formats — and compare segments directly in your survey results dashboard. Unlimited responses on the free tier means you can actually reach significance instead of running underpowered tests. [Build a free survey](/free-survey-builder) [Calculate sample size](/tools/sample-size-calculator) ## FAQs ### What does the p-value actually mean? The probability of observing a difference at least as extreme as yours, assuming the two groups are truly identical. A p-value of 0.03 means there's a 3% chance you'd see this big a difference purely by random sampling. ### My result is barely significant — should I trust it? Borderline results (p between 0.04 and 0.05) replicate poorly. If the decision matters, run a confirmation test. The closer p is to 0.05, the less robust the finding. ### Why use a pooled standard error? Under the null hypothesis (the two groups are identical), the best estimate of the shared rate comes from pooling both groups. Using a pooled SE produces a more accurate test than calculating SE separately for each group. ### What if my sample is unbalanced? The z-test handles unequal sample sizes fine — n\_A and n\_B don't need to match. But heavily unbalanced designs are less efficient: you get the most statistical power for a fixed total budget when n\_A ≈ n\_B. ### Is this a frequentist or Bayesian test? Frequentist. It returns a p-value, not a posterior probability. If you want "probability variant B is actually better," you need a Bayesian A/B testing tool — the math is different and the answers to different questions. ## Related - [Sample size — full definition](/glossary/sample-size) - [Margin of error — full definition](/glossary/margin-of-error) - [Confidence interval — full definition](/glossary/confidence-interval) - [Response rate — full definition](/glossary/response-rate) - [Sample Size Calculator](/tools/sample-size-calculator) - [Margin of Error Calculator](/tools/margin-of-error-calculator) - [NPS Calculator](/tools/nps-calculator) - [CSAT Calculator](/tools/csat-calculator) **SpaceForms** is **The only mainstream survey builder offering unlimited responses on a true free-forever tier.** - Unlimited responses (free) - AI survey in 30s - 36.1% mobile completion - Voice + chat modes SpaceForms Free online form & survey builder. Unlimited responses. No credit card required. [Start free](/try) ### Templates by category - [Education](/templates/education) - [Customer Experience](/templates/customer-experience) - [HR & People](/templates/hr-people) - [Healthcare](/templates/healthcare) - [Market Research](/templates/market-research) - [Events & Training](/templates/events-training) - [UX & Product](/templates/ux-product) ### Popular templates - [Net Promoter Score (NPS)](/templates/survey/nps-net-promoter) - [Customer Satisfaction (CSAT)](/templates/survey/csat-post-purchase) - [Customer Effort Score (CES)](/templates/survey/ces-customer-effort) - [Brand Awareness](/templates/survey/brand-awareness) - [Employee Engagement](/templates/survey/employee-engagement-annual) - [Teacher Feedback](/templates/survey/teacher-feedback) - [Product-Market Fit](/templates/survey/product-market-fit) ### Guides & comparisons - [Free survey builder](/free-survey-builder) - [Survey & research glossary](/glossary) - [Resource library (all guides)](/library) - [NPS calculator](/tools/nps-calculator) - [CSAT calculator](/tools/csat-calculator) - [Sample size calculator](/tools/sample-size-calculator) - [Best free survey makers](/articles/free-survey-maker) - [Qualtrics pricing review](/articles/qualtrics-survey) - [CES vs CSAT](/articles/ces-vs-csat) - [Qualtrics vs SurveyMonkey](/articles/qualtrics-vs-surveymonkey) - [Google Forms vs SurveyMonkey](/articles/google-forms-vs-surveymonkey) - [SpaceForms vs SurveyMonkey](/articles/spaceforms-vs-surveymonkey) - [Jotform vs SurveyMonkey](/articles/jotform-vs-surveymonkey) - [Typeform vs Jotform](/articles/typeform-vs-jotform) - [Microsoft Forms vs Google Forms](/articles/microsoft-forms-vs-google-forms) - [Survey definition & types](/articles/survey-definition) - [Survey question types](/articles/survey-questions) ### Company - [Home](/) - [Blog](/blog) - [Sign in](/auth) - [Try the builder](/try) © 2026 SpaceForms. Free online form & survey builder. Built with love by the SpaceForms team.