How a geo experiment works
A geo experiment splits a market into regions and treats them differently. Some regions carry on unchanged; others receive the change — more budget, a new channel, or nothing at all. The regions are chosen so that both sets behaved similarly before the test, then compared across the test window. Because location is something ad platforms target precisely, the split is easy to enforce and hard for a visitor to cross by accident.
The comparison is not simply sales in one set against sales in the other. What you actually want is an estimate of what the test regions would have done had nothing changed, built from the control regions and from how the two sets normally move together. The gap between that estimate and what really happened is the effect.
Why geo experiments matter
Some questions cannot be answered inside an ad account. Cookie-level measurement misses people who saw an ad on one device and bought on another, in a shop, or over the phone. A geo experiment sidesteps all of that, because it measures whole regions and total outcomes rather than tracked individuals. That makes it one of the few honest ways to value a channel that produces offline or phone-based enquiries — the normal situation in Nepal, where a lot of buying still finishes on a call or in person.
It is also the practical version of a holdout test for a business that cannot split its audience technically. Turning a city off is something any advertiser can do.
Where geo experiments go wrong
Badly matched regions ruin the result before it starts. Two places with different seasons, different competitors or a different mix of customers will diverge on their own, and the experiment will confidently report an effect that is nothing of the sort. Matching has to be done on past behaviour, not on a feeling that two cities are similar.
The other failures are practical. Media spills across boundaries, especially television, radio and national social pages. Someone changes a price, opens a branch or launches a promotion in one region halfway through. And too few regions on each side leaves the whole conclusion at the mercy of one unusual city having an unusual month.
How to run one
Match the regions on their own history first, over a period long enough to see how they normally track each other. Set the length of the test from the buying cycle rather than from convenience, and hold everything else — creative, pricing, other channels — constant while it runs. Write down beforehand what result would change the plan, so the finding is not renegotiated afterwards.
Keep the design simple. A clean comparison between two clearly separated sets of regions, read on one agreed metric, is far more useful to a business than an elaborate model nobody in the room trusts. And treat the answer as belonging to this market, this offer and this period; rerun it when any of those change materially.
