What incrementality measures
Reporting platforms count the conversions they touched. Incrementality asks a harder question: of those conversions, how many would have happened without the advertising? The gap between the two is the part your spending genuinely caused, and it is the only part worth paying for.
The test is always a comparison against a group that did not see the campaign. That can be a formal holdout, where a slice of the audience is deliberately excluded, or a geographic split where some regions run the campaign and comparable regions do not. Several platforms offer built-in conversion lift studies that work the same way. Switching a campaign off for a clean period and watching what happens to total sales is the crude version, and it still teaches you something when the market is stable.
Note what is not being measured. Incrementality is not an attribution model. Attribution divides credit between the touchpoints on a converting journey; incrementality asks whether the journey needed the advertising at all.
Why incrementality matters
Some campaigns are very good at appearing next to purchases that were already going to happen. Branded search, remarketing to people with items in the basket, and prospecting that keeps reaching existing customers all report well and can add far less than the report implies. Cutting them may cost nothing; scaling them may buy the same sales twice.
It matters more as tracking weakens. With consent controls, modelled data and journeys that cross devices, platform reports increasingly describe an estimate. A holdout measures outcomes rather than cookies, so it stays honest when the tracking underneath it does not.
Common mistakes with incrementality
The most common is running a test too small or too short to say anything. If the difference between the exposed and held-out groups is smaller than the ordinary week-to-week noise in your sales, the result is a coin toss dressed as evidence — the same trap as calling an A/B test early.
The rest is contamination. Changing budgets, creative, prices or promotions mid-test makes the result unreadable. Seasonal swings, festival trading and a competitor’s launch can drown the effect if the period is chosen carelessly. And a single test answers a single question at a single moment; treating the answer as a permanent property of a channel is a stretch.
How to act on it
Test where the stakes justify the disruption: your largest line of spend, or the campaign you most suspect of taking credit for other people’s work. Decide before you start which result would change your decision, and how long you are willing to hold the test open. A test with no pre-agreed consequence tends to end in a debate about methodology.
Then act on the answer properly. A channel that proves less incremental than it reports does not have to be switched off. Often the right response is to move budget within it, cap frequency, or stop bidding on the audiences most likely to convert unaided. Keep a written record of what each test showed and when, because the value only compounds if next year’s plan can read last year’s evidence.