A channel can report conversions it observed and still leave the counterfactual unanswered. A credible geo test starts with the comparison before the media change.
Key moments
- 0:00 Platforms together claim more conversions than the business made, so which ones are incremental?
- 0:37 Some channels, like TV or podcasts, can cause sales that click attribution never reports.
- 0:57 A geo test compares parts of the country that historically moved together.
- 1:41 Pre-test checks show whether the groups match and how much data a clear result needs.
- 2:05 Compare the measured change with what the platform reported to calibrate it.
- 2:51 The readout gives a confidence range; if it’s too wide, you may need a bigger test.
What this video covers
A credible design starts before the media change: assess viable markets, define treatment and holdout conditions, protect against interference, and set the decision rule. The readout should include uncertainty, applicability, and a practical recommendation—not just a lift number.
Read the related guide: Incrementality Testing for DTC Brands