Your Ad Accounts Are Grading Their Own Homework
Why platform ROAS and business performance stopped being the same number, and the 14-day test that tells you which one to trust.
A founder sent me three screenshots last month. Meta: 4.1 ROAS. Google: 3.6. His Shopify dashboard for the same 30 days: 2.2 blended. He wanted to know which one was broken.
None of them. That's the uncomfortable part.
Every ad platform is graded by an exam it wrote itself. Meta will count a purchase as its own if the buyer saw an ad in the previous seven days — saw, not clicked. Google will count the same purchase if there was a click in the previous thirty. TikTok has its own view-through logic. One customer makes one purchase, and two or three platforms each write it into their own revenue column, in good faith, according to rules they published openly.
So the reports don't disagree. They answer different questions. The problem is that founders are making one decision — where the next hundred thousand dollars goes — using three answers to three different questions.
THE FOUR-MINUTE VERSION
Open every ad platform you spend on. Pull the revenue each one reports for last month. Add them up. Now open your actual store analytics and pull real revenue for the same window.
In most accounts I've opened over the last five years, the platforms collectively claim somewhere between 20% and 60% more revenue than the business actually made. Not because anyone is cheating. Because the same sale is being counted more than once, and because view-through attribution credits impressions that may have changed nothing at all.
If your number comes back over 100%, you now know something important: the reports you've been steering by describe a business larger than yours.
WHY THE OVER-REPORTING ISN'T THE REAL PROBLEM
Inflated numbers are annoying. What they cause is expensive.
Here's the sequence I watch play out. A brand reviews channel performance. Meta shows the strongest ROAS. Budget shifts toward Meta. The following month Meta's reported ROAS holds up, so more budget shifts. Six months later, spend is up 60% and blended MER is flat or down, and nobody can point at the decision that caused it.
What happened is that Meta looked strongest partly because it was claiming conversions that Google, email, and organic search also touched. Moving money toward it didn't buy more customers. It bought more attribution. And when the channel eventually ran out of borrowed demand to take credit for, the ceiling arrived all at once and looked like a creative problem.
This is the single most common way I see a healthy account quietly become an unhealthy one. It doesn't start with bad creative or bad targeting. It starts with a reallocation decision made on a number that was never designed to support one.
WHY BUYING BETTER ATTRIBUTION SOFTWARE DOESN'T END THIS
Get Abhishek Pinninti’s stories in your inbox
Join Medium for free to get updates from this writer.
I've used most of the tools. Some are genuinely good and I still recommend them for directional reads. But it's worth being clear about what they are.
Every attribution model — last click, data-driven, multi-touch, media mix modelling — is a guess about a counterfactual. It's trying to answer a question no dataset contains: would this person have bought if the ad hadn't run? You cannot observe that. You can only estimate it, and every estimate carries the assumptions of whoever built the model.
A holdout is different in kind. It doesn't estimate the counterfactual. It creates one.
THE TEST THAT SETTLES IT
Pick one channel you're unsure about. Split your market into two matched sets of geographies — matched on revenue, order volume and seasonality, not on population. Cut the channel's spend by 50% in one set. Hold it flat in the other. Change nothing else for fourteen days: no new creative, no landing page tests, no promotions.
Then compare total revenue between the two sets. Not the channel's reported revenue — total revenue, from your own store.
Three things can happen. Revenue in the cut geos falls roughly in line with the spend reduction, which means the channel was doing close to what it claimed. Revenue barely moves, which means most of what that channel reported was demand that would have arrived anyway. Or revenue falls harder than the spend cut, which means the channel was doing more than its report showed — this happens with upper-funnel TikTok more often than people expect.
Any of those three outcomes is worth more than a quarter of dashboard analysis, because it's evidence rather than inference.
THE NUMBER THAT CAN'T LIE TO YOU
Blended MER — total revenue divided by total advertising spend across every channel — has one virtue nothing else has: there is no window, no model, and no platform incentive inside it. It is arithmetic on two facts.
It is a blunt instrument. It won't tell you which creative to kill or which audience to expand. It doesn't need to. Channel reports are diagnostics; blended MER is the scoreboard. The mistake almost everyone makes is treating a diagnostic as a scoreboard, and then wondering why the score keeps disagreeing with the bank account.
WHAT I'D DO THIS WEEK
Reconcile platform-reported revenue against real revenue for last month, and write the ratio down where your team can see it. Put blended MER at the top of your weekly report, above every channel number. Then pick your least certain channel and start a fourteen-day geo holdout.
None of that costs anything. All of it changes what your next budget meeting is actually about — which is not which platform wins, but how one budget should be split across three of them.
That's the harder question, and it's the one I'll take apart next week.
—
I'm Abhishek. I've spent five years running paid acquisition for e-commerce brands, and I write about where the money actually goes. If you want a second pair of eyes on your own numbers, my inbox is open.
