"Our rep says we should run a geo test. What is that, exactly?"
So you've heard the phrase. Maybe from an agency or a platform rep. Everyone seems to agree it's the "real" way to measure ads. But nobody has explained what it is, or whether it fits your question.
Let's fix that.
What is geo testing?
Here's the plain version. A test is where you take a section of the population and you don't show them the ads. Then you compare them to a section of the population where they do see the ads.
In a geo test, those sections are places, like states or metro areas.
Think of it this way. Does a rising tide lift all boats? A geo test is where you test the tide and the boats to find out.
Google's own researchers have written about it. They describe geo experiments as regions randomly assigned to a test or control condition, with ads targeted by location. They call the approach "conceptually simple."
It is. Getting it right is the hard part.
How does a geo test work?
There are four parts.
- Pick regions. Test regions (the ones you change) and control regions (the ones you leave alone).
- Check the pre-period. Before anything changes, the two groups should track each other in your sales history.
- Make the change. You do this with location targeting. Google Ads, for example, lets you target or exclude locations. Google notes that location targeting isn't 100% accurate in every situation.
- Read the result. Compare what the test regions sold with what the control regions say they would have sold.
There are two common designs. In a holdout, you turn a channel off in some regions. In a scale-up, you add spend in some regions.
One labeling quirk. In a holdout, the regions you change go dark, and I call them the holdout. The regions you leave on are what I call the treatment. Providers label this differently, so check which is which before you read anyone's report.
Example (hypothetical, simulated numbers): Say I pause a paid social channel in holdout regions that make up about 20% of sales. Over the test window, the treatment regions predict the holdout would have sold $200,000. It actually sells $190,000. That $10,000 gap is my estimate of what the channel added there, if that gap is bigger than the test's MDE range.
Numbers in this example are simulated for illustrative purposes. They are not a client result or a benchmark.
Geo test vs. A/B test vs. platform lift study
An A/B test or a platform lift study usually splits people. It relies on cookies, logins or device IDs to know who saw what. A geo test splits places.
That matters because people switch devices, clear cookies, visit family and tell friends. A region holds all of that inside it. And a geo test reads sales in your own record, not a platform's report.
Where both are possible, I prefer geo tests over user-level lift.
What can a geo test tell you, and what can't it?
It can estimate what a channel added to total sales, in that period, at that spend level. It can also show effects on other channels, if you can measure their sales by region. Compare each other channel's sales in the test and control regions to see whether pausing one channel changed what another one sold. Total sales stay the bottom line.
It can't hand you a permanent number. One test is one data point. Be careful turning a single result into a fixed rule for every season and spend level. Customers change. So do your offers.
What a good test can do is anchor your other measurement. Test results can be used to calibrate an MMM (Google Research). More on what a test can calibrate in an MMM.
A geo test answers one question well. It doesn't answer every question forever.
How long does a geo test need to run?
Start with the math. Work out the minimum detectable effect (MDE), as a range, and power over a proposed test length, using your own sales history. There's no universal MDE. It comes from your data.
Then check that length against your customers. How long does it take someone to go from seeing an ad to buying? I want several of those cycles inside the window.
My rule of thumb is a 4-week minimum. Shorter tests give only a directional read, and a higher chance of a null result, so they fit only if you accept both.
You'll often hear "4 to 6 weeks" for a geo test. Treat that as a place to start the conversation, not a rule. A brand that sells a cheap refill people buy on impulse and a brand that sells a mattress people research for weeks or months shouldn't run the same test length. Use the planner to plan how long your test needs, and see why lift tests end too early.
How do you choose test and control regions?
You have two main options. Matched markets use real regions that have tracked each other in the past. A synthetic control builds a comparison from a weighted mix of other regions. Free, open-source tools exist for this, like Meta's GeoLift (synthetic control) and Trimmed Match (paired regions), a research library from Google.
For a matched-market test, I usually look for one or two control markets. Here's how to choose between matched markets or a synthetic control. Before you launch, run three checks on your own sales history, with free code.
When is a geo test worth running?
When the answer would change what you do. That usually means one of these:
- More than one meaningful channel is competing for budget.
- Your platform conversions add up to more than your actual orders.
- A big spend change, or a pause, is on the table.
- You can't trust cookie or device data to tell you who saw what.
That last one is common now. When user-level tracking is shaky, a geo test still works, because it never needed to know who anyone was.
If you're thinking of switching off search, here's a geo holdout before you pause paid search.
Before anything launches, agree on the action. If the team won't say, "If this works, we will do it," the question may not be worth testing yet. And a positive holdout can lead to the next test: a scale-up.
Test the decision, not the dashboard.
If you're not sure your channel can be tested, talk through whether a geo test fits your question. Or start with the duration planner.
FAQs
Is a geo test the same as a matched market test?
Close. A matched market test is one kind of geo test. It compares your test regions with real regions that have tracked them in the past. Other geo tests use a synthetic control instead, built from a weighted mix of regions. Either way, you're comparing places, not people.
How much does a brand need to spend to run a geo test?
There's no universal spend threshold. It depends on how big your spend change is next to your daily sales volume, and how noisy those sales are. A small test cell, or a lot of day-to-day swing, needs a bigger change or a longer test. Run the feasibility checks on your own data first.
Can a geo test measure TV or podcasts?
Yes, where you can buy or hold out the media by region. Last click undervalues upper-funnel media. A podcast ad is rarely the last click, and TV never gets a click at all. A geo test reads total sales instead, which is why it fits these channels. See incrementality testing for how I approach it.
Big picture
Geo testing sounds fancy. It isn't. You change ads in some places, leave others alone, and compare sales.
The fancy part is doing it right: sizing it for your data, running it long enough, and agreeing on the decision first. No model owns the truth. But a well-run test gets you a lot closer!
Need help? Speak with me here.