Before You Call a Geo Test Feasible: 3 Pre-Launch Checks (With Code)

“Can we get a geo test on paid social before planning starts?”

So you say yes, pick a few DMAs, and schedule the change.

Twelve weeks later the readout lands. The interval is so wide it supports a budget increase and a cut at the same time. Nobody changes the budget.

But you could have seen that coming. The answer was in your daily sales history, BEFORE anyone touched a campaign.

How do you know a geo test is feasible before you launch it?

Check your daily sales history before anyone touches a campaign. Three checks decide it. First, does the control track the test geos? The notebook looks at correlation and scaled-control MAPE, then runs a dry run that should show no lift. Second, can the spend and test length you have detect the change you care about? Third, will leadership act on the result? If one check fails, fix it first. The notebook ends with GO, FIX, or DON'T TEST.

How to use this guide with the code

Everything on this page runs in a free notebook: geo-test-pretest-checks on GitHub. It’s a small Python module plus one Jupyter notebook, and it works from a file you export yourself.

1. In GA4, export daily purchase revenue (or transactions) by date by DMA as a CSV. There’s no GA4 connection and no credentials. Just the file.
2. Open notebooks/pretest_checks.ipynb and point it at your export.
3. List your test geos and your control geos. One or two controls is typical.
4. Replace every value marked EXAMPLE (test length, planned change, target iROAS, margin, budget) with your own.
5. Run it top to bottom. The last cell prints GO, FIX, or DON’T TEST, with the reasons.

python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
jupyter lab notebooks/pretest_checks.ipynb

It ships with a synthetic sample (20 made-up DMAs, 455 days, no client data), so you can kick the tires first! Every example number below comes from that sample.

What it doesn’t do. It doesn’t search for markets, simulate power across designs, or read out lift after the test. You choose the geos. For those jobs, use the open-source tools these checks sit in front of: GeoLift from Meta, Trimmed Match from Google, and Meridian GeoX from Google. Credit to those teams for the methods this builds on.

The code is MIT licensed. The docs are CC BY 4.0.

Name the decision first

Before you open the notebook, write one sentence. What will this test decide?

“Measure incrementality” isn’t a decision. “Keep, cut, or scale paid social next quarter” is. (The notebook’s checklist has a decision field for exactly that sentence.)

Then write the smallest change in sales that would make you decide differently. That number is your planned change in Check 2, so everything below gets sized to it.

Check 1: Does your control track the test geos?

During the test, the control has one job. It shows you what the test geos would have sold without the change.

If it can’t do that job on history, when you already know the answer, it won’t do it during the test either. (Whole markets or a weighted blend? See matched markets vs. synthetic control, or the short video.)

Do the two cells move together?

Check correlation between the test cell and the control cell. Do it on daily data and on weekly data. Check each one over your full history, and again over the most recent stretch (about two times your test length). That’s four numbers.

Here’s how I read it. Above 90% is acceptable. Above 95% is what I’d rather see. And weekly correlation can’t be weak. If it is, pick different controls.

What if daily looks weak but weekly looks fine?

That’s a pass, with a warning. Read the test on weekly numbers. But weekly reads give you a seventh as many data points, so plan a test at least 1.5x as long as you’d planned, rounded up to whole weeks.

How far off is the control, once you scale it?

Your control cell and test cell won’t be the same size. So multiply (“scale”) the control so its sales line up with the test cell. Fit that scale factor on a stretch of history no shorter than the test.

Then measure the miss. MAPE (mean absolute percentage error) is the average percentage gap between the test cell and the scaled control, week by week. The notebook shows WAPE (weighted absolute percentage error) next to it.

• Under 10%: strong.
• 10% to 15%: acceptable.
• 15% to 20%: caution. Not a failure, but your MDE goes up and the result gets less certain.
• Over 20%: weak.

Does it read zero when nothing changed?

Think of a bathroom scale. Before you trust it, you check that it reads zero with nobody on it.

A dry run (or “placebo test”) is the same idea. Hold back the last stretch of your history, about as long as the test and never shorter than two weeks. Fit the scale factor and the noise model on the earlier history only. Then have the control predict the held-back window.

Nothing changed in that window. So the “lift” should be about zero. I count it as a pass only when both of these are true:

1. The placebo lift is smaller than the upper end of your MDE range (more on that in Check 2).
2. Its 90% interval includes zero.

Example: In the sample data, I put Example DMA 03 and 08 in the test cell and Example DMA 05 and 12 in the control, for an 84-day test. Daily correlation came in at 84.9% (weak), weekly at 93.5%, and weekly MAPE at 2.5%. So it passes on weekly data only. The dry run showed a placebo lift of +0.9%, with a 90% interval of -1.5% to +3.3%, against an upper-end MDE of 3.1%. Both conditions hold. PASS.

(The dry run’s MDE runs a little higher than Check 2’s, because it learns the noise without the held-back weeks.)

If the comparison finds a lift when nothing happened, it will find one during the test too.

Check 2: What will it cost to see the lift you care about?

Your test cell and control will never match perfectly. So any real lift has to be bigger than the normal gap between them, or you can’t tell it apart from noise.

People call that threshold the “minimum detectable effect” (MDE). And once you have it, the budget math is one line:

required spend = MDE × baseline treatment sales over the test window ÷ target iROAS

The notebook sizes the MDE from your own noise, sized for 90% confidence, 80% power, one-sided (it only looks for lift in the direction you planned). It reports the MDE as a range, low to high. I make decisions on the upper end.

Don’t have the export yet? The geo-test duration planner gives you a quick directional range in the meantime.

What’s a power score?

It’s the chance your test picks up your planned change, at your planned length. I want 80% or better. Under 80%, the notebook tells you the shortest length, in whole weeks, that gets there. And if no length up to a year gets there, change the cells or the hypothesis.

Here’s the sample data again, same cells, same 84 days, with two different planned changes:

Example: My 84-day MDE range is 2.7% to 2.9%. Baseline sales in the test geos over the window come to 754,500. At an example target of 2.5x iROAS, the spend I need is about 8,200 to 8,600. If I plan a 3.0% change, the power score is 83%. But correlation only passed on weekly data, so the notebook still recommends 126 days. GO, with a warning.

Same data, but I plan a 2.0% change instead. Power drops to 58%, and the notebook recommends 175 days. FIX.

Example values from the repo’s synthetic sample. Not a benchmark.

A one-point difference in the plan moved the test by seven weeks! If you change the length, re-run the checks at the new length to confirm.

Holdout or heavy-up?

• Holdout (turning spend off in the test geos). Your baseline is recent sales with the spend still running, so it includes the sales that spend drives. The test reads a drop. So the spend you switch off, times its iROAS, has to be at least the MDE times the baseline.
• Heavy-up (adding spend). The formula gives you the extra spend on top of business as usual. Put in your current spend in those geos as a sanity check. If the extra is bigger than what you spend now, the iROAS may not hold at that scale.

Would a win make money?

Compare your target iROAS (on revenue) with breakeven, which is 1 ÷ gross margin. At a 55% margin, breakeven is 1.82x. If your target sits below breakeven, a “win” still loses money. Rethink the target before you test.

Lengths are whole weeks. Weekly reads run Monday to Sunday, and a week with a missing day gets dropped.

Size the test to the change you’ll actually make.

Check 3: Will anyone act on the result?

This check has no math in it. It might matter most.

Who makes the call, and have they agreed to it?

Sit down with the decision maker before launch. Ask what budget numbers they’d be comfortable with. Ask whether they’d roll a winner out nationally.

If the honest answer is no, don’t run this test. Test something they would act on instead. That’s the DON’T TEST verdict.

What will you do with each result?

Write a stepped plan while nobody knows the answer yet. I start with a band of ±20% around the iROAS I expect. You can widen or tighten it based on your appetite for change and your volume.

• Lands inside the band: act on the decision.
• Lands well above it: make a digestible change (say, a measured spend increase) and retest.
• Lands well below it: make a digestible change and retest, or run a different test.

Example: With a 2.5x target, my band runs from 2.0x to 3.0x. Anything in that range, I act on.

What does a null result mean?

It means “not proven.” It doesn’t mean “doesn’t work.” A geo test proves or disproves one effect at one spend level. It doesn’t hand you a response curve.

If you plan to feed the result into an MMM later, here’s what an incrementality test can calibrate in an MMM, and what it can’t.

A result only matters if someone agreed, in advance, to act on it.

Pre-commit the readout

Check 3 says what you’ll DO with each result. This part says how you’ll read it. Write it down before the campaign changes:

• The outcome: the same sales column you ran the checks on.
• The test and control geos, and the tool you’ll read the result with (GeoLift, Trimmed Match, or Meridian GeoX).
• The test window, in whole weeks.
• How you’ll show uncertainty. An interval, not just the midpoint.

The easiest record is the notebook itself. Save the executed copy, with its verdict, MDE range, and stepped plan, next to the test plan.

Swap the metric after a bad result, and the test can’t tell you anything.

Reading the verdict

• GO: all three checks pass on your data, with your assumptions. Read the warnings anyway.
• FIX: something has to change first. Different controls, a longer test, a bigger change, or answers from leadership.
• DON’T TEST: the business wouldn’t act on the result. Pick a different question.

Got a GO? Next, put that design into a full test plan.

When a check fails

Example: The sample data includes one bad control on purpose. Example DMA 17’s sales level drops partway through the history. Paired alone against Example DMA 03, it shows 50.6% daily and 38.7% weekly correlation and a 22.5% weekly MAPE. The verdict is FIX: choose different controls.

Before you trust the inputs

Can you use GA4 for this? For a comparison between two cells, yes, with one check first. Compare GA4’s sales with your backend orders in each cell. If GA4 captures a similar share of orders in both cells, its tracking gaps hit both cells about equally, and the comparison holds up. If the shares differ a lot, sort that out first.

The notebook also handles a few things for you:

• Missing days stay missing. They’re reported, never counted as zero sales.
• (not set) geos and total rows get dropped and reported.
• Duplicate date and geo rows get summed.

And a few it can’t see: spillover between geos, overlapping promotions, and regional events. Rule those out yourself.

FAQs

Sources and limits

These thresholds are how I read the numbers, from practice (I’ve designed and run 70+ incrementality tests). They’re guidance, not rules, and every one is a setting you can change in geo_pretest/checks.py. The calibration was checked on synthetic data, not on your data. The MDE is an estimate from past noise, not a promise. And a GO says the test is worth running. It doesn’t say the result will be positive.

• geo-test-pretest-checks, Eric Brinker, Brinker Analytics (2026). Code MIT, docs CC BY 4.0, synthetic data only.
• GeoLift (Meta), Trimmed Match (Google), and Meridian GeoX documentation (Google), reviewed October 15, 2026: the design and analysis tools these checks sit in front of.
• A Time-Based Regression Matched Markets Approach for Designing Geo Experiments, Tim Au (2018).
• Robust Causal Inference for Incremental Return on Ad Spend with Randomized Paired Geo Experiments, Aiyou Chen and Tim Au (2022).
• Periodic Measurement of Advertising Effectiveness Using Multiple-Test-Period Geo Experiments, Jon Vaver and Jim Koehler (2012).

The big picture

Spend an afternoon with your own sales history before any media changes. Check the control. Price the lift. Get the decision and the readout on paper.

Then launch the test you already know can give you an answer!