Picture this: your MMM says to move a big chunk of budget out of one channel and into another.
That’s a material decision. So it deserves a serious challenge before you act on it. And one good way to challenge it is to run more than one MMM approach on the same question.
The point isn’t to average the answers. It isn’t to let the majority win, either. You want to know whether several credible ways of looking at your data tell a story that hangs together.
Isn’t this just a vote?
No.
Marketing data isn’t a clean lab. Channels often rise and fall together. Promos and seasonal demand land on top of your media. And customers see several messages before they buy.
So every MMM has to make choices. How do you structure the data? How do you transform spend? How much carryover and saturation do you assume? And what can your history actually tell apart?
Running several workable model setups (each one is called a “specification”) is a model-risk review. One approach may find a relationship more clearly than another. Another may show that a reading that looked precise depends on a fragile assumption.
If the views broadly agree, you have stronger grounds to act.
If they don’t, the honest answer may be that you can’t score that channel with confidence yet.
So when two models do disagree, what do you check first?
What should you compare?
Start with the evidence each model used
I have one strong caution here: a lot of model “disagreements” are really input problems.
Check that the models use the same outcome definitions. The same spend taxonomy. The same promo and seasonality context. The same time period.
If the gap comes from different inputs, that’s a reconciliation task. It isn’t a deep insight about method. Line up the inputs, then compare again.
Look at ranges and sensitivity
Pull the iROAS range for each channel, not just the point estimate.
A range, or a wide confidence interval, is information. It tells you how sure the model is.
Then ask: does this channel look good under only one narrow carryover or saturation setting? If so, the recommendation should say that out loud.
Explore the plausible ranges using your data and what you know about the business. And don’t treat a convenient setting as if it were a universal DTC fact.
Turn each reading into what you’d do
For each channel, write down what changes if the model is right about the direction:
- Would a lower-funnel channel need a stricter last-click target?
- Would an upper-funnel channel earn a controlled increase?
- Would the answer be “don’t score it yet”?
The comparison should help your team decide what to do. A longer technical appendix doesn’t do that.
What does it mean when models disagree?
It’s common. And it’s not a verdict.
Disagreement shows up a lot when the historical signal is weak or the channels are highly correlated. It doesn’t automatically mean a channel failed. It doesn’t mean one framework is broken, either.
It may mean a relationship you can see in a focused experiment can’t be pulled out cleanly from the wider history. Or the history just hasn’t moved enough to separate two channels.
For a tested channel, a well-designed lift study is the causal anchor for the conditions it tested.
A model like Google Meridian can take that result as a known constraint while it estimates the channels you haven’t tested. Then a second approach can ask whether the broader history supports the same story on its own. The test won’t make every doubt go away. But it gives the models firmer ground to stand on.
And what if a big channel gives you conflicting readings, you can’t test it, and there’s no clear way to learn more? Don’t turn that doubt into false precision. Call it provisional. Keep the question open. And avoid making a large decision you can’t undo on a number your team can’t explain.
When is one model enough?
Check three things.
- Is the conclusion stable across reasonable assumptions?
- Is the decision reversible, or low stakes?
- Do the recommendations match the other evidence your team already has?
If yes, one well-documented approach may be enough. More models add cost, more work to interpret, and more room for confusion.
Some teams also need one operating number so they can move fast. That can be sensible. As long as the team stays clear about what that number can’t see.
So the question isn’t “should every brand run three MMMs?” It’s “how much challenge does this decision deserve?”
Use several workable views where the stakes justify it. Use testing where a specific causal question needs an anchor. Then hand your team a decision system it can rerun and question, instead of a black-box answer it just has to accept.
FAQs
Do multiple MMMs make the answer more accurate?
Not automatically. They make it more likely that big assumptions, data limits, and conflicting readings show up before your team acts on them.
What if two models disagree about a channel?
First line up the inputs and look at the uncertainty. Then turn the disagreement into a hypothesis, a temporary guardrail, or a test plan. Don’t pick a winner just to get back to a single score.
Can a lift study replace MMM?
A lift study is stronger for the specific change it tests. MMM is still useful for the broader portfolio, and for finding the next question worth testing.
When that next question needs a geographic holdout, the guide to matched markets versus synthetic control covers the comparison design.