By Dr Grace Kite

The Bermuda triangle of marketing measurement, and how to fly out of it

Share

If you’ve ever read anything about measuring the effect of marketing, you’ve probably come across the triangle of unified methods, introduced by Google in 2019.

It says you should use MMM, experiments, and attribution together to get a view of what works: MMM takes care of the big picture, attribution gets more granular, and experiments validate both.

It sounds like a perfect recipe, and in the years since 2019 it’s become accepted best practice.

But in 2025, it’s bad advice. Attribution, now without a full set of 3rd party cookies, gives flat out wrong results, and no amount of cross-referencing can save a badly set-up experiment or MMM.

Worst of all, trying to manage three methods this way prevents marketing people from taking action. The methods are different, and one of them is always wrong, so you can’t see the clear route ahead, and you’re never comfortable enough to act.

This is the Bermuda triangle of marketing measurement, where actions that would improve your media plan disappear and are never found again.

And there’s really no need to be stuck with it. In October, the IPA and Melt collective introduced a better framework, called MESI, that actually gets results.

Media-owner-advises-industry-to-rely-on-its-own-research shocker

The triangle of unified measurement put Google’s own tools front and centre as best practice. Their attribution at one corner and their in-platform experiments at the second.

Source: Measuring Effectiveness, Three Grand Challenges, Google, 2019

Relying on a media owners’ answer about how the advertising they sell pays back is never a good idea. But this particular set of advice has got worse as time’s gone on.

In the attribution corner, third-party cookies are now so patchy that multi-touch and data-driven attribution is not much better than last click. That means all attribution is now heavily biased towards ads that appear at the last minute before a purchase: in particular, search.

In 2019, if you bought an online video ad for a pair of trainers, you could be pretty confident of tracking the vast majority of people that saw it and then bought the trainers.

In 2025, you might place the same ad, but a huge slug of people that block 3rd party cookies wouldn’t appear in your figures. So, you’d massively underestimate what the ad does for sales, and probably buy more search instead.

In the experiments corner, a method which academics and scientists point to as the most robust can also end up biased.

Experiments work by showing your ad to one group of people and comparing sales in that group to a set of people that didn’t see it; the difference is the effect of the ad. Sounds foolproof, right?

But platforms choices in building the tests can lead to wrong answers. They can suffer from something called “divergent delivery”: the algorithms serving ads to people in the test group that it reckons are likely to respond, while in the comparison group, it’s not people the algorithms fancy, it’s anyone in the wide target audience.

According to a recent academic paper, this leads to experiments finding too big an effect of the ad under study.

It’s not surprising that Google recommended a framework that allowed it to evaluate its own ads, but it is surprising that so many have bought into the advice, for so long.

Instead: Moving from messy to MESI

The MESI framework set out in the diagram below is different.

It recognises that attribution is fatally flawed, and drops it completely. And, although it still recommends using modelling/MMM and experiments, it doesn’t require these methods to agree.

Instead, MESI matches methods to the questions that each is well suited to answer. Modelling is for questions about what generally worked in the past, and experiments are for questions on specific things that haven’t been tried yet.

Instead, MESI matches methods to the questions that each is well suited to answer. Modelling is for questions about what generally worked in the past, and experiments are for questions on specific things that haven’t been tried yet.

But the real joy in MESI is that it prompts you to brings answers to all these questions together in a forward-looking simulation. This step gets teams talking about the future in an informed way, and it leads to next steps that everyone’s happy to implement.

The acronym, MESI, stands for sequential steps that make up a recipe:

  • The M is for modelling – econometrics or MMM. You do that first to get an overview of what works based on what you’ve done before.
  • E is for experiments – properly designed by an independent expert. You do this next to test things you haven’t tried yet.
  • S is for simulate – combining results into forward looking scenarios that forecast total sales and profit. You do this with everyone in the room, including finance.
  • I is for implement – agreeing on the future you want and taking the steps to get there. You agree on a plan B too, and know the criteria for when you need to trigger it.

Our most sophisticated clients at magic numbers are using this framework, nestling all the steps into powerful technology developed by our parent company Analytic Partners.

And it works. For example, the pan-European retailer who’s able to wargame their future using MMM results and the findings of good quality experiments, all in the same simulation.

All of their markets know how media plans ladder up to the C-suite’s desired outcomes, and they implement those plans, monitoring how well each decision works and tracking the revenue improvements they get.

It’s how the global C-suite got behind marketing as a driver of growth, and it’s how they got €220m in additional revenue without increasing spend.

Flying out of the Bermuda triangle

More than 50 ships and 20 planes have disappeared in the real-life triangular bit of sea and air between the Bahamas, Bermuda, and Puerto Rico since the 1800s.

No-one ever died because of bad marketing. But if we’re counting money, the loss to businesses from the Bermuda triangle of marketing measurement is much, much bigger.

Millions of marketing departments have been stuck in an endless loop of mismatched analytics and conflicting reports, all tied to an ineffective framework that fails to translate into real business impact.

Attribution is dead, experiments have their limitations, and numbers that come with ulterior motives get you nowhere.

MESI changes that. It delivers alignment across teams, integrates finance into the conversation, and ensures that marketing decisions aren’t just data-driven, but business-driven.

So don’t be a lost ship in 2025 and don’t let your business be a plane that goes down and vanishes. Make this the year you adopt marketing measurement that actually works.

 

© 2025 Website &AGENCY Privacy Policy
X