Attribution and incrementality are not the same thing, and confusing them is one of the most expensive mistakes in performance marketing. Attribution tells you which campaigns received credit for conversions. Incrementality tells you which campaigns caused them. The gap between those two things is often large, and it consistently points to the same channels: brand search, retargeting, and any channel that reaches people who are close to buying anyway.
Imagine you run a brand search campaign. Your branded terms capture people who already know you and are searching for you by name. They convert at a very high rate. Your attribution model credits the campaign with those conversions and reports a stellar ROAS. But most of those people would have found you organically if the paid ad had not been there. The campaign captured demand it did not create. Its true incremental value is much lower than what attribution reports.
The same dynamic plays out with retargeting. People who visit your site and then see a retargeting ad are more likely to convert than cold audiences. But some of them were going to convert regardless of the ad, they were already far enough along in their decision. Attribution gives the retargeting campaign full credit. Incrementality testing reveals the true marginal contribution.
Without incrementality measurement, budget tends to flow towards the campaigns that look best on attribution, which are often the ones that are least incremental. You end up spending heavily on demand capture while underinvesting in demand creation.
Geo holdout tests split your target geography into matched regions. You run campaigns normally in the test regions and stop them (or significantly reduce them) in the control regions. After a sufficient period, you compare conversion rates between the two. The difference is your incremental contribution. This is the most robust method but requires enough geographic variation in your audience to run cleanly.
Conversion lift tests are Meta's built-in incrementality testing product. They create a holdout within a campaign audience, a randomly selected group who do not see your ads, and compare conversion rates between the exposed and holdout groups. The output is an estimated incremental conversion rate. It is less flexible than a geo test but simpler to run for advertisers without the geographic data structure to do geo splits.
Budget pause tests are the simplest and least rigorous. You pause a campaign for a defined period and watch what happens to overall conversions. If you pause a brand search campaign for two weeks and total conversions barely change, the campaign was capturing conversions that would have happened anyway. This approach is confounded by seasonality and other variables, so interpret results with caution.
Select matched regions, areas that historically show similar conversion rates and demographic profiles. Avoid regions with very different characteristics or those affected by local seasonality. Run the test for long enough to smooth out weekly variation, typically three to four weeks minimum. Compare conversion rates (not absolute volumes, which may differ by region size) between test and control.
The incremental lift is the difference in conversion rate between the two groups. Multiply that by your total conversions in the test period to get an estimate of how many conversions would not have happened without the campaign. That is your incremental ROAS denominator.
Before making major budget decisions. If you are planning to significantly increase or decrease spend on a channel, knowing its true incremental contribution is essential. Scaling a channel that looks good on attribution but delivers low incrementality is expensive. Cutting a channel that looks weak on attribution but has high incrementality is an even more expensive mistake.
Brand campaigns and retargeting are the highest-priority targets for incrementality testing because they are the most likely candidates for attribution inflation. Run a test on your brand search campaign once, the results are often surprising.
If you want help designing or running an incrementality test for your campaigns, get in touch.