So you’ve decided to run a geo test. You’ve picked the markets where the media will change. New to the idea? Start with what geo testing is.
Now comes the harder question. What do you compare them to?
A matched market test compares your test markets with real control markets that have tracked them in the past. A synthetic control builds a weighted blend of several markets instead.
A real market is simpler to explain, and it leaves your other markets free. A synthetic control can fit better when no single market tracks the test markets. Which one wins? Back-testing on your history should decide. Not a preference for either label.
What does every geo test have to answer?
A geo holdout changes ad exposure in part of the country. You might pause a channel in selected markets, try a new strategy, or test a different spend level.
But the change alone doesn’t prove lift.
You still need a believable answer to one question: what would have happened in those markets if the media hadn’t changed? That unseen alternative is the counterfactual. Matched markets and synthetic controls are two ways to build it. Both can be useful. Neither one is automatically the better answer for every brand, channel, or geography.
What is a matched market?
It’s a real place standing in for another real place.
A matched-market design pairs your test geography with one or more real control markets that behaved alike in the past. The appeal is simple. If Seattle has reliably tracked your test markets, Seattle can stay a real, observable comparison while the media changes somewhere else.
But being concrete isn’t proof. “Similar on average” isn’t enough.
Markets can look alike over a long period and still split apart during the spikes and valleys that matter to a test. A weather event, a local promo, an inventory issue, a competitor’s push, or a different demand pattern can break a pair that looked great. Real markets keep all the mess of the business. That’s their value. It’s also their limit.
A pair that matches on average can still break when it counts.
What is a matched market test, step by step?
A matched market test changes media in some markets and compares their sales with real markets that tracked them before the change. The gap that opens up is your estimate of lift. You’ll also hear it called a geo lift test. Same family: a geo test that reads the lift in total sales between test and control regions.
Here’s how it runs:
- Pick candidate test markets. Where can you actually change the media?
- Find real markets whose sales tracked them. Designated Market Areas (DMAs) help here. There are more than 200 in the U.S., so you have more room to find a fit than with 50 states.
- Back-test the pairing before launch. Check it on untouched history, not just the stretch you picked it on.
- Change media in the test markets only. Then compare sales.
One note on labels. In a holdout, the test markets go dark, and I call them the holdout. The markets that stay on are what I call the treatment. Providers label this differently, so check.
Free tools exist for both. Meta’s open-source GeoLift is built on synthetic control methods. Trimmed Match, a research library from Google, analyzes paired geo experiments.
What is a synthetic control?
Take several geographies. Give each one a weight. Blend them into a benchmark that looks like your test markets’ past behavior.
That’s a synthetic control. Instead of using one real market as the control, it uses a weighted mix of other markets to estimate what your test markets would likely have done without the change. It can get a closer fit to history when no single real control market is good enough.
The tradeoff is capacity, and how easy it is to explain.
A synthetic control may draw on a broad share of the country. That leaves fewer untouched markets for another test running at the same time. And a lower-volume business may already need a large holdout to detect a meaningful result, which makes several comparisons impractical. That’s a feasibility finding. It’s not a reason to force a test.
When is a synthetic control unstable, and how do you choose donor markets?
A close-looking chart isn’t proof.
A synthetic control is a weighted historical comparison. It isn’t a literal second group of markets. And it doesn’t promise what a national rollout would do.
What does unstable look like?
- The result depends on one or two markets. Tools like GeoLift report the weight each market gets. It’s normal for a few markets to carry most of it. So re-run the comparison leaving out each heavy market in turn. If dropping one moves the result a lot, a local event there could be moving your whole comparison.
- The fit is tight on the history it was built on and loose on history it never saw. That’s why you back-test on untouched history and run placebo checks before launch.
How do you choose donor markets?
Donor markets are the pool the blend is built from. Good donors aren’t exposed to the media you’re testing, and they aren’t in another test at the same time. They also shouldn’t be shaped by a regional demand pattern your test markets don’t share. Think of snowblower sales that follow storms in one region.
More donors isn’t automatically better. A big pool makes it easier to fit the past by chance. Keep the pool to markets that behave like your test markets, and use the leave-one-out check above. DMAs give you more good candidates to choose from.
What if it’s still unstable?
Fall back to a real matched market if one tracks well enough. Or accept a wider range on the result, and say so. A wide range isn’t automatically a reason to do nothing. If both ends point to the same move, you may be able to act.
What you shouldn’t do is call a design credible because the chart looks close. There’s no fit threshold that works for every brand.
If your markets won’t match cleanly, talk through which comparison fits your data. Or run three pre-launch checks, with free code, and plan how long it should run.
How do you choose and check the comparison before launch?
Work backward.
Start with your candidate test markets, then go back through the historical data. Where both are workable, test a real matched-market option and a synthetic-control option.
For each one, ask: is it the right size? Does it track the test markets’ pattern after scaling? And does it stay close most of the time, not just on average?
MAPE (mean absolute percentage error) is one useful way to see how far a comparison drifts from the line. It catches a control that averages well but is too jumpy to predict the test markets reliably.
| Design | How the control is built | Where it fits | Main risk |
|---|---|---|---|
| Matched markets | One or more real markets that have historically tracked the test markets | A real market tracks the test markets closely; you want a comparison that is simple to explain and leaves other markets free for other tests | A local event (weather, promotion, inventory, competitor activity) breaks the pair; a one-of-a-kind market may have no real match |
| Synthetic control | A weighted blend of several markets, fit to the test markets’ prior behavior | No single real market predicts the test markets well enough | Draws on a broad share of the country, leaving fewer untouched markets for concurrent tests; harder to explain |
Pre-test baseline testing
A control that looks good on paper still has to pass a test. But this one happens before any media changes.
Run baseline tests on the candidates. Confirm that each control fits the test markets and the other controls in the pre-period. Check the fit with measures like MAPE and the correlation between test and control. Those show how well the controls predicted the test series before launch. Then run placebo checks.
How many controls? As many as your data allows. Each cell has to be big enough for a good read. Each needs enough data. And you need enough comparable baseline to show each control fits.
In practice, that’s usually one or two controls at most.
So once a control passes, should you stop there?
Check more than one view
Where your volume and test capacity allow it, check several comparisons before launch:
- a matched-market view,
- a synthetic-control view,
- and the broader national trend.
A result that holds up across these views is harder to dismiss. And if the readings differ, the gap itself tells you something. It can show whether the test is sensitive to a fragile choice of control.
How long should it run, and what else do you set up?
Longer than you might think, sometimes.
Checking the comparison also means planning the execution. Define which geographies get or lose media. Check for channel bleed. Choose the outcome and the observation window based on how long your customer takes to decide. And set the decision rule before you see the result.
Four to six weeks in market is a starting point for the conversation. It’s not a rule. The power math needed to confirm a non-null result sets the length, adjusted for how the tactic drives action. An upper-funnel brand video for a high-AOV product with a long consideration cycle may need MUCH longer than six weeks.
How does this work with an MMM?
They do different jobs. And each one feeds the other.
An MMM can point to a portfolio-level hypothesis. Maybe a channel is adding less (or more) incremental value than last-click reporting suggests. A geo test can check one specific causal question under set conditions.
The lift result can then calibrate the model. And the model helps you find the next question worth testing. Neither method removes the need to understand the other.
For how much weight a test result should carry, read what an incrementality test can calibrate in an MMM.
FAQs
What is a matched market test?
It’s a geo test that compares markets where media changed with real markets where it didn’t. The control markets are chosen because their history tracked the test markets closely. If the two keep moving together until the media changes, the gap that opens up afterward is your estimate of lift.
What is a geo lift test?
It’s a geo test that reads the lift in total sales between test and control regions. Matched markets and synthetic controls are two ways to build the control side. The result is an estimate with a range, not an exact number. New to this? Start with what geo testing is.
Should we always use synthetic control?
No. It can give an excellent fit to history. But a real matched market may be more stable or more practical. Test capacity and back-testing on your history should decide.
Can a small DTC brand run a geo test?
Possibly. But it may need a large holdout or a longer observation period. If the design can’t credibly detect a change that matters for the decision, a different measurement step is better.
Should a geo test use more than one comparison?
When your data and capacity allow it, yes. Agreement across a matched-market comparison, a synthetic control, and the national trend can make you more confident in what you learned.
How many control markets do you need?
As many as the data allows, which is usually one or two at most. Each cell needs to be big enough for a good read and have enough data. And there has to be enough comparable baseline to show each control fits the test markets and the other controls. The Geo Test Signal Planner gives a directional read, and Before You Call a Geo Test Feasible covers the rest of the pre-launch checks.
How do you know a control is good enough?
It has to pass pre-test baseline testing. Back-test it on untouched history. Check the pre-period fit with measures like MAPE and the correlation between test and control. And run placebo checks. A control that averages well but drifts in the spikes and valleys isn’t good enough.
How long should a matched market test run?
Four to six weeks is a starting point for the conversation, not a rule. The power math needed to confirm a non-null result sets the length, adjusted for how the tactic drives action. Slow-acting upper-funnel work for a high-priced product can need much longer. Use the geo-test duration planner for a directional read.