Nobody Checked the Broken File
Match rate is calculated by the vendor being evaluated, on a denominator it picks, and it is rarely audited. The market still treats it as a benchmark.
I’ve been on the vendor side of several competitive match tests. One still bothers me.
Several identity providers received the same file and were asked to report a match rate. We got zero. Not a weak result. Zero. Not one record resolved.
We spent most of a week testing every hashing method, normalization rule, and field-preparation assumption two engineers could put on a whiteboard. The same pipeline matched every other file we processed that month.
We told the sender the file was broken. Their answer: we probably did not know what we were doing.
A week later, they confirmed the file was bad. Something upstream had corrupted it before distribution. They sent a clean version. We reran it and landed in the high seventies, roughly where we expected.
Another vendor had already reported a match rate of about 70% on the broken file. Once the corruption was confirmed, nobody asked how.
There may be an innocent explanation. Maybe that vendor received a different build. Maybe it used a key the corruption did not touch. Maybe it had a pre-existing crosswalk around the damaged fields. We don’t know because nobody asked — not the sender, not the buyer who liked the number, and not us, if I’m honest. We had spent a week proving our own system worked. At the time, that felt like enough.
That is the whole problem in one story. Low match rates get interrogated. High ones end the meeting.
I could fill this article with match rates from four companies. I won’t. Some are confidential; all are stale. A match rate belongs to one file, one key mix, one graph, and one moment. It does not travel. Yet vendor-reported rates are routinely treated as portable benchmarks.
The number nobody can check
CIMM’s 2023 television match rate study gave five identity providers the same two TV datasets and let each use its normal process. There was no common instruction on which keys or household IDs to use. Reported rates ranged from 63% to 86%. Against location data, they ranged from 31% to 69%.
That is not a scoreboard. It is what happens when several providers use different rules under the same label. The consequence was not theoretical. CIMM estimated that provider choice alone could swing reported ROI on the same campaign by as much as 20%.
More important, the study did not test accuracy. The authors wanted to, but they could not obtain an independent dataset linking IP addresses to physical addresses. A widely cited television match rate study measured coverage. Accuracy was out of reach.
Two years later, they got the validation set.
In November 2025, CIMM and Go Addressable published work conducted by Truthset, a data-accuracy vendor that scored 938 million IP-to-postal pairs and 5.6 billion IP-to-email pairs from six major data providers against ground truth supplied by two ISPs and a virtual MVPD. IP-to-postal links were accurate 13% of the time. IP-to-email links, 16%. The publicly posted deck labels all results preliminary.
Averages hide the shape. IP-to-postal accuracy ran from 4% to 18% across providers, so the best one measured was roughly four and a half times more accurate than the worst. And 77% of all IP-to-postal linkages scored under 10% accuracy. That is not a distribution with a low mean. It is a pile of near-worthless links with a thin tail of good ones.
Then the finding that should have been the headline. Only 6.4% of IP-to-postal linkages were reported by more than one provider. For IP-to-email, 2.8%. That is not a 6.4-point spread between reported rates. It is the share of links that any second provider corroborated at all. Ninety-four percent of IP-to-postal links and 97% of IP-to-email links appeared in exactly one provider’s data, over the same 90-day window.
The supporting detail is no better. Data providers reported an average of 3.2 IP addresses per postal address; the ISPs that assign them reported 1.2. One real household in the study showed 11 IPs during the window, 9 of them wrong. Providers used timestamp fields to mean different things: first seen, last seen, opt-in date, or batch version. An earlier Truthset and CIMM study had put average email-to-postal linkage accuracy at 51%.
Truthset illustrates how the error compounds by chaining the stage averages: 13% at IP-to-postal, 51% at email-to-postal, then a demographic-accuracy rate on top. The deck renders the result as three cents of every dollar still on target. That is a modeled cascade across separate studies, not a measured finding that 97% of CTV spending is wasted, and it assumes the errors at each hop are independent. If a provider that gets one hop right tends to get the next one right too, the real figure is better than three cents. It is an illustration of direction and magnitude. Treat it as one.
A separate 2026 study from Adstra and InterMedia reported that residential-IP targeting run without Adstra’s new scoring layer reached the intended geography only 23% of the time. The research accompanied the launch of that product, so treat it as directional corroboration rather than independent validation.
These errors sit underneath targeting, frequency capping, and attribution across the entire channel.
Match rate of what?
Before calling a match rate good, ask what it is a rate of. At least six choices sit underneath the headline number. Vendors make those choices differently. Most don’t disclose them by default.
What entered the denominator. A file arrives with 10 million rows. Some fail basic hygiene: malformed addresses, missing fields, records that cannot be hashed, duplicates. Suppose 6 million records match. That is 60% of the 10 million you submitted, but 73% of the 8.2 million that survived cleanup. The graph did not improve. Only the denominator changed. Either number can be defensible. Only one tells you what share of your customer base you can reach.
What counts as matched. A match to any node in the graph is a looser claim than a match to an identifier you can actually activate, which is looser still than a match to a device seen in biddable supply during the past 30 days. The first definition produces the largest number and often the least useful one. In CTV, the distance between “we hold this household” and “we have seen this household recently” can explain the entire gap between two vendor quotes.
What unit is being counted. Person, household, or device. A household-level result presented without a qualifier can make person-level coverage look much larger than it is. In CTV, household is often the most defensible unit.
How fan-out is counted. One CRM record resolves to six devices in one home. Is that one match or six? Against which denominator? A vendor that counts device-level matches against a person-level file can report more than 100%, and some do.
When the match was measured. Onboarding, activation, and attribution can produce different answers. CIMM flags this problem in CTV because campaigns rely heavily on IP. If the address is reassigned before the attribution join, the household that saw the ad can disappear from the analysis. A different household enters with the inherited address, and its non-response gets charged to the campaign.
Which keys carried the result. Hashed email, IP, MAID, and postal data produce different rates and different error profiles. A 78% rate built mostly on IP is not equivalent to a 78% rate built mostly on authenticated hashed email. With IP-to-postal accuracy benchmarked at 13%, the key mix often explains most of the result, yet it rarely appears in the deck.
Two vendors can quote 62% and 81% on the same file and both be telling the truth. Until you know how each made these six choices, you are not comparing anything.
Coverage is not accuracy
Match rate answers a real question: how much of this universe connects to something else? If a brand brings 10 million records and only 2 million can be activated, that constrains reach, pacing, and budget. That matters.
The mistake is treating coverage as proof that the connections are correct. Accuracy is a separate measurement, and almost nobody produces it.
Graph A matches 600,000 of 1 million records and is 98% accurate against a validation set. That yields 588,000 correct matches and 12,000 false ones. Graph B matches 850,000 at 80% accuracy. That yields 680,000 correct matches and 170,000 false ones.
Graph B produces 92,000 more correct matches — and 158,000 more errors.
For broad prospecting, that trade may be acceptable. For purchaser suppression, household-level measurement, high-value retargeting, or frequency control, 170,000 false positives are a liability you paid to create. At a $35 CTV CPM, bad matches are not rows in a table. They are spend, frequency, creative exposure, optimizer feedback, and attribution credit pointed at the wrong homes.
The pitch deck celebrates the extra 25 points. The operator should ask how much error came with them.
How a probability becomes a yes
An identity graph is a set of links. Every link has a source, an age, and a confidence level.
Some paths are short and strong. An authenticated login tied to a household and then to a CTV device is about as good as this gets. Others are chains of inference: an IP pattern leads to a household cluster, which leads to a device, which leads to an inferred viewer. A path that was defensible in March may be weak by August.
In a well-built graph, confidence falls as signals weaken, links age, and paths get longer. Then the nuance disappears. Everything collapses into one flag: matched.
Links go stale for ordinary reasons. A household moves. Roommates split. A student leaves for school. A houseguest signs into a streaming app on someone else’s TV and never signs out. An IP relationship can hold for a year and then quietly stop. That churn is one reason IP-to-postal accuracy sits in the low teens.
A false negative costs reach, which in a scale-constrained campaign is a real cost. A false positive creates a different risk: it tells the system to act on a link that may be wrong. It can target the wrong home, suppress the wrong buyer, credit the wrong exposure, and train the bidder on the wrong pattern.
Get Prashant Chaudhary’s stories in your inbox
Join Medium for free to get updates from this writer.
That is where this stops being a media waste problem. Feed an optimizer enough false-positive conversions and it starts learning from events that look real only because the identity layer made them joinable. The model sees a household, an exposure, and a conversion in what appears to be a causal sequence. Then it looks for more inventory that resembles the pattern.
One bad match is a reporting error. A hundred thousand are training data.
More attribution, same lift
A brand measures a completed CTV campaign through a conservative graph. Fewer exposed households join to the conversion file. The attribution report shows 10,000 matched conversions. A holdout shows 3% incremental lift.
The analyst reruns the same exposure and conversion files through a looser graph. More exposed households join. The report now shows 16,000 matched conversions.
The holdout still shows 3%.
Nothing changed in the business. The graph created more joinable events, not more demand. That is how match rate can raise attribution without raising incrementality. It is the same mechanism behind CIMM’s finding that provider choice can swing reported ROI by up to 20%.
The holdout still has to be designed carefully. If the control group, exposure group, and outcome matching all depend on the same weak graph, the lift result inherits the same bias. The control cell is not outside the identity problem.
The vendor grades its own homework
This is how a 13% IP-to-postal accuracy benchmark can coexist with match rates in the 70s.
The party being evaluated calculates the match rate. It chooses the denominator, selects the key mix, defines what counts as a match, decides whether to report against the submitted file or the surviving file, and usually discloses none of it. Its renewal depends on the result.
Most mature media metrics have some external check. Viewability gets accredited. Impression counting gets audited. Brand-safety claims get reviewed by third parties. Match rate is self-reported and self-defined. Until late 2025, there was no public, large-scale benchmark of commercial IP-to-household linkage accuracy.
The research says this part out loud. The CIMM report’s list of known limitations opens by noting that providers often prioritize match rate and scale over recency and accuracy, and its closing analysis observes that without standard benchmarks, vendors are trusted by default rather than by proven accuracy.
This does not require villains. Put reasonable people inside that structure and you get drift, not necessarily fraud. Definitions loosen one step at a time in the direction that helps. A lower-confidence tier gets folded into the headline rate because a competitor did it first. Nobody has to decide to mislead. The number simply stops meaning what it once meant, and no outside process exists to notice.
The buyer is trapped too. A planner who insists on high-confidence-only activation cuts the addressable universe, under-delivers, takes the client call, and loses the next RFP to the vendor with the larger number. The market punishes quality at every step. Asking buyers to be more disciplined won’t fix the incentive.
Accuracy is not the only quality dimension, and a conservative graph can be precise and still selective. CIMM’s earlier television study found that matched samples skewed older and higher-income, because identity graphs overrepresent households that are established and stay put.
That is a coverage-bias problem, not an argument against measuring accuracy. Providers should report both: accuracy by confidence tier, and the composition of the records that fall out of the matched sample. Are the links correct is one question. Who is easiest to link, and who gets left out, is a different one.
Six questions before you trust the number
When someone shows me a match rate, I ask these before discussing whether the number is good:
- What exactly is the denominator, and what share of the submitted file was dropped before calculation?
- Which keys produced the match, and how much of the result depends on IP?
- What share is deterministic, and what share is inferred?
- How old are the links, and when were they last confirmed against an independent source?
- Can low-confidence matches be excluded separately from suppression, attribution, and bidding?
- Which use cases should never use the low-confidence tier?
The last question is the most revealing because almost nobody asks it. A serious partner can answer directly: this tier is acceptable for broad prospecting but not purchaser suppression; that tier is useful for planning but too weak for closed-loop attribution.
When every match is treated as usable in every workflow, the product is chasing coverage at the expense of decision quality.
Can the product use the answer?
Confidence tiers are useless if the product cannot select them. Real-time bidding has genuine latency limits. Planning tools, suppression workflows, and attribution screens do not. If those products cannot isolate a confidence tier, that is a product decision, not a timeout problem. “The UI does not expose it” should not end the conversation.
The standards are starting to catch up. IAB Tech Lab added ID provenance fields to OpenRTB 2.6 in 2024 to signal who inserted an identifier, who matched it, and by what method. The Data Transparency Standard pushes disclosure on recency and provenance while making clear that disclosure is not a quality grade.
But disclosure matters only when decision systems use it. Provenance sitting in a bidstream field that no bidder reads is documentation, not control.
The same rule applies to clean rooms. A clean room can govern a join. It cannot make the join true. A weak match stays weak inside stronger privacy controls.
The better test
Ad tech treats an unknown user as lost revenue. Unmatched households leave holes in a deck. Nobody wants to say, “We cannot resolve this person with confidence.” Nobody wants to hear it either.
Sometimes unknown is the most accurate state the system has. A platform that refuses to use a weak match for suppression is doing better product work than one that quietly converts uncertainty into activation. The goal is not to resolve everyone. It is to know when a match is strong enough to act on.
Match rate still matters. A sudden drop can mean something broke. A rate that is too low can stop a campaign from scaling. A large gap between two partners on the same file is worth investigating. CIMM also shows that wide spreads can appear even when providers are running their normal production processes.
But match rate is a coverage claim, and the industry reads it as a truth claim.
Here is the test I would run before choosing or renewing a provider. Randomly split the source file before matching, then build treatment and holdout cells inside each half. Resolve one arm with high-confidence links only and the other with the full graph. Hold budget, inventory rules, creative, and outcome measurement constant. Then compare frequency, CPA, attributed conversions, suppression quality, and incremental lift, using an outcome source independent of the tested graph where you can get one. If the full graph wins on attribution but not on lift, the metric improved. The business did not.
Sources
Primary documents first; secondary coverage after. Paywalls noted.
- CIMM, Go Addressable and Truthset, The Challenges of Using IP Address for Audience Identification and Measurement (public report deck, PDF; preliminary results), November 2025. Announcement and summary.
- CIMM, Television Data Match Rate and Bias Study, September 2023. Full report (PDF).
- Adstra and InterMedia Advertising, Joint Research on Identity Quality in CTV Targeting, July 2026. Also covered in Adweek (paywalled).
- IAB Tech Lab, ID Provenance added to OpenRTB 2.6, September 2024.
- IAB Tech Lab, Data Transparency Standard.
Prashant Chaudhary has spent two decades building advertising and streaming products, including identity and measurement systems at 4INFO, Cadent, and Yahoo, and CTV monetization platforms at Amagi. He is an inventor on US Patent 10,419,883 for cross-device identity resolution. The opening match test is described without naming the participants, client, or data.
