# What Is Incrementality Testing for Modern Marketers Incrementality Testing is a controlled experiment that reveals which conversions your marketing actually caused. Source: https://amplitude.com/en-us/explore/experiment/incrementality-testing --- ###### What incrementality testing means and how it measures true marketing impact # What Is Incrementality Testing for Modern Marketers Incrementality Testing is a controlled experiment that reveals which conversions your marketing actually caused. Learn how to measure real lift and optimize spend with Amplitude. Table of Contents - Incrementality testing answers a core question in marketing: which outcomes are caused by campaigns and which would have happened anyway. The method separates [correlation from causation](https://amplitude.com/blog/causation-correlation). Reported [conversions](https://amplitude.com/explore/metrics/conversion-rate-guide) often include baseline behavior unrelated to media. Incrementality testing isolates the caused outcomes by comparing exposed and unexposed audiences. Privacy changes limit user-level tracking across channels. [Controlled experiments](https://amplitude.com/explore/experiment/product-experimentation) provide a rigorous way to measure causal impact without relying on personal identifiers. Browse this guide - [What is incrementality testing?](#definition) - [Why incrementality in marketing matters now](#benefits) - [How to measure incrementality and calculate lift](#measure-and-calculate) - [Incrementality calculation formula](#incrementality-calculation-formula) - [Reading incremental revenue in analytics platforms](#reading-incremental-revenue-in-analytics-platforms) - [Channels and use cases for incrementality testing in marketing](#channels-and-use-cases) - [Paid social and display campaigns](#paid-social-and-display-campaigns) - [Search ads and shopping feeds](#search-ads-and-shopping-feeds) - [In-product feature rollouts with holdout groups](#in-product-feature-rollouts-with-holdout-groups) - [Five actions for reliable marketing incrementality tests](#actions) - [Turn lift into continuous growth with unified analytics and experimentation](#unified-analytics) ## What is incrementality testing? Incrementality testing is a controlled experiment that measures the true causal impact of marketing campaigns. It compares outcomes between two groups: one exposed to your campaign (treatment group) and one that isn’t (control group). The control group shows what happens without your marketing, while the treatment group shows what happens with it. The difference between these groups is your incremental lift—the results actually caused by your campaign. Here’s what each term means: - **Lift:** Additional outcomes caused by marketing activity above baseline behavior - **Control group:** Audience withheld from the marketing campaign - **Treatment group:** Audience exposed to the campaign being tested - **Causal effect:** Changes in outcomes directly attributable to the campaign Traditional [attribution methods](https://amplitude.com/explore/digital-marketing/marketing-attribution-guide), like last-click, often credit campaigns for conversions that would have happened anyway. Incrementality testing reveals which results your campaign actually created versus which it just touched along the way. ## Why incrementality in marketing matters now Most marketing platforms overstate their impact. [Facebook](https://www.facebook.com/business/), [Google Ads](https://ads.google.com/), and other channels report conversions that include people who would have converted without seeing your ads. Privacy changes make this problem worse. iOS updates limit tracking through apps, while browsers restrict third-party cookies. These changes create gaps in attribution data, making platform reports less reliable. Marketing incrementality testing solves this by measuring actual cause and effect: - **True impact measurement:** Proves campaigns cause specific outcomes rather than just correlating with them - **Budget optimization:** Identifies which channels create new customers versus capture existing demand - **Future-proof approach:** Works without user-level tracking or cookies For example, a retailer might see high conversions from branded search ads. However, an incrementality test could reveal that 80% of those conversions would have happened through organic search anyway. The real incremental value is just 20%—dramatically different from what the platform reports. ## How to measure incrementality and calculate lift Incrementality measurement compares outcomes between test groups using a simple formula. The calculation shows how much your campaign actually moved the needle above baseline behavior. ### Incrementality calculation formula The basic formula is: (Treatment Group Results - Control Group Results) / Control Group Results. This gives you the percentage lift caused by your campaign. For example, if your treatment group had a 5% conversion rate and your control group had a 4% conversion rate, your incremental lift is 25%. You can apply this formula to any metric: - [Conversion rates](https://amplitude.com/track/conversion) - [Revenue per visitor](https://amplitude.com/explore/metrics/what-arpu-how-calculate) - [Customer lifetime value](https://amplitude.com/templates/customer-lifetime-value-dashboard) (CLTV) - [Retention rates](https://amplitude.com/track/retention-rate) ### Reading incremental revenue in analytics platforms Platforms like Amplitude display incrementality results across multiple dimensions. You can see the lift broken down by audience segment, geography, time period, and campaign element. The key metrics to track include: - **Absolute lift:** Raw difference in outcomes between groups - **Percentage lift:** Relative improvement over baseline - **[Statistical significance](https://amplitude.com/explore/experiment/statistical-significance-guide):** Confidence that results aren't due to chance - **Net incremental revenue:** Total additional revenue caused by the campaign Amplitude's experimentation features calculate these automatically and show [confidence intervals](https://amplitude.com/explore/experiment/confidence-intervals) to help you understand the reliability of your results. ## Channels and use cases for incrementality testing in marketing Different marketing channels benefit from incrementality testing in specific ways. The approach works best for channels where the relationship between exposure and conversion isn't immediately apparent. ### Paid social and display campaigns Social media and display ads often influence customers over time rather than driving immediate clicks. Traditional attribution misses this delayed impact. Incrementality tests for these channels typically measure: - Brand awareness lift through surveys or branded search volume - [Conversion rate](https://amplitude.com/explore/experiment/conversion-rate-optimization) changes over 30–90 day windows - [Customer acquisition cost](https://amplitude.com/templates/cost-acquisition-cost-dashboard) for truly new customers ### Search ads and shopping feeds Search campaigns capture demand that might exist organically. An incrementality test reveals how much additional traffic and revenue your ads actually create. Common tests include: - Pausing ads in specific geographic regions - Testing different keyword match types - Comparing bid levels to measure true incremental ROAS ### In-product feature rollouts with holdout groups Product teams use incrementality testing to measure feature impact. A holdout group continues using the old experience while the test group gets the new feature. This approach measures effects on: - [User engagement](https://amplitude.com/explore/analytics/digital-experience-analytics) and session depth - Conversion through key funnels - Long-term retention and customer lifetime value ## Five actions for reliable marketing incrementality tests Running effective incrementality tests requires careful planning and execution. These actions help avoid common mistakes that can invalidate your results. ### 1. Right-size your groups for statistical power Your test groups need enough people to detect meaningful differences. Too small, and you won’t see significant results even if your campaign works. Too large, and you’re withholding campaign exposure unnecessarily. Use your baseline conversion rate and expected lift to calculate [minimum sample sizes](https://amplitude.com/explore/experiment/power-analysis). Most tests need at least 1,000 people per group to detect a 10% lift confidently. ### 2. Isolate one variable per test Test one campaign, creative, or channel at a time. Multiple changes make it impossible to know what caused any differences you observe. For example, don't launch new creative and increase budgets simultaneously. Run separate tests for each change so you can measure their individual impact. ### 3. Control for seasonality and external factors Holidays, competitor actions, and market changes can skew results. Run tests during stable periods or ensure both groups experience the same external conditions. Account for factors like: - Seasonal shopping patterns - Competitor promotional campaigns - Economic news that might affect buying behavior - Product launches or PR events ### 4. Connect lift to long-term customer value Immediate conversions don't tell the whole story. Track how incremental customers behave over time to understand true campaign value. Use [cohort analysis](https://amplitude.com/templates/cohort-analysis-dashboard) to measure: - Repeat purchase rates for incrementally acquired customers - Customer lifetime value differences between the test and control - Retention patterns over 90+ day periods ### 5. Automate reporting with integrated platforms Manual analysis introduces errors and delays insights. Platforms like Amplitude automate incrementality calculations and provide real-time results as your test runs. Integrated tools maintain consistent user assignment, track exposure accurately, and calculate [statistical significance](https://amplitude.com/glossary/terms/statistical-significance) automatically. ## Turn lift into continuous growth with unified analytics and experimentation The most effective approach to incrementality testing integrates measurement with your existing analytics stack. This eliminates data silos between campaign management and results analysis. Point solutions require manual data exports and custom analysis for each test. A [unified platform](https://amplitude.com/explore/experiment/experimentation-platform-guide) like Amplitude connects [user behavior tracking](https://amplitude.com/templates/user-behavior-dashboard) with experimentation features, so you can measure incrementality using the same data that powers your other analytics. | | | | | | ---------------- | -------------- | ---------------------------- | ----------------------------- | | **Approach** | **Setup Time** | **Data Consistency** | **Ongoing Maintenance** | | Point solutions | Weeks | Manual reconciliation needed | High—custom analysis required | | Unified platform | Days | Automatic data alignment | Low—automated reporting |   With unified analytics and experimentation, you can run continuous incrementality tests across channels and campaigns. Results feed directly into budget allocation decisions without waiting for separate analysis cycles. This approach makes marketing incrementality testing a standard part of campaign optimization rather than a special project. Teams can quickly identify what's working, scale effective tactics, and eliminate wasteful spending. [Try Amplitude for free today](https://app.amplitude.com/signup) to start measuring the true incremental impact of your marketing campaigns with integrated analytics and experimentation tools. ## Explore related content [ExploreWhat Is Digital Marketing?](/explore/digital-marketing/what-is-digital-marketing)[Blog PostFrequentist vs. Bayesian: Comparing Statistics Methods for A/B Testing](/blog/frequentist-vs-bayesian-statistics-methods)[ExploreP Values: What They Are and How To Calculate Them](/explore/experiment/p-values)[Blog PostWhat is A/B Testing? How it Works and When to Use It](/blog/ab-testing)[Blog PostMarketing Forecasting 101: Steps, Benefits, and Data to Use](/blog/marketing-forecasting)[ExploreWhat Is A T-test? ](/explore/experiment/t-test)[ExploreWhat Is A Multi-Armed Bandit? Full Explanation](/explore/experiment/multi-armed-bandit)[ExploreWhat Is Smoke Testing in Software Development](/explore/experiment/smoke-testing)