A new study found that gaps and errors in advertising data can push marketing budgets in the wrong direction, especially when they affect customers who make a purchase.
An advertising campaign can make money and still be switched off because the dashboard says it is failing.
New research from the Marketing + Media Alliance and data company LiveRamp shows how this can happen. The problem begins when an advertising system cannot reliably connect the person who saw an ad with the person who later bought the product. The findings are explained in the full research report.
A customer might see an ad on a phone, research the product on a laptop and complete the purchase in a shop. To measure the campaign, the business must work out whether those separate records belong to the same customer.
When records are matched incorrectly, a purchase may be linked to the wrong person. When records are missing, the sale may appear to have happened without any advertising at all.
Both problems can make an effective campaign look much weaker than it really is.
A $1.50 return appeared as just 43 cents
The researchers tested the problem by deliberately removing advertising records and linking some of them to the wrong people.
The study’s attribution tests used 1.9 million advertising impressions involving 147,941 users and four publishers. The data also included 12,956 transactions. The researchers changed the data in controlled ways to see how the reported results differed from the known results.
In one deliberately severe test, a campaign reached 30% of the audience, but only half of the customer matches were correct.
The campaign was set to produce a 25% increase in conversions and return $1.50 for every dollar spent. After the incorrect matches were introduced, the measurement system reported a return of just 43 cents per dollar.
A profitable campaign therefore appeared to be losing 57 cents for every dollar spent. Its measured increase in conversions also fell from 25% to approximately 6.8%.
The study does not claim that advertising campaigns normally have customer-matching accuracy of only 50%. The researchers described this as an illustrative scenario designed to show how the problem works.
The roughly 70% fall in measured return was a result from this simulation. It should not be treated as an estimate of how much advertisers generally lose because of poor customer matching.
The report says its findings show the possible direction and scale of the problem, but more testing with real campaigns is needed.
Small gaps can cause surprisingly large mistakes
Another part of the research produced an even more useful finding.
The researchers first removed 20% of advertising impressions at random. Despite losing a large amount of data, the measurement model still placed the four publishers in the correct performance order.
They then ran a different test. This time, the missing records were concentrated among customers who completed a purchase.
Only about 1% of the total impression data was removed, but the model’s publisher ranking reversed. Advertising channels that appeared to be performing well fell behind channels that were actually less effective.
This means the location of a data gap can matter more than its size.
A business may believe its data is reliable because 99% of the records are present. That remaining 1% can still cause serious problems when it mainly includes buyers, subscribers or other valuable customers.
The model’s usual accuracy score also failed to reveal the problem. In the test involving missing data from purchasers, the score improved even as the channel ranking became less accurate.
A normal dashboard check could therefore make the model look healthier while its business recommendation became worse.
More advertising decisions are being automated
The problem becomes more important as advertising platforms make more decisions automatically.
Google says its Smart Bidding system uses artificial intelligence to adjust bids in each advertising auction based on expected conversions or conversion value.
The LiveRamp study did not test Google Ads or any other specific automated bidding product. However, its findings point to a wider risk.
When an automated system is told that one channel produces more sales than another, it may direct more money toward that channel. Incorrect customer matches can give the system the wrong signal.
Automation may then allow a measurement error to influence thousands of spending decisions before a person reviews the results.
Businesses using automated advertising should therefore pay close attention to how conversions are recorded and matched, rather than relying only on the final return shown in a dashboard.
What marketers should check
Before cancelling a campaign, a business can compare the number of impressions reported by each advertising platform with the number appearing in its own measurement system.
It should also ask measurement providers about precision. A provider may report that it can match a large percentage of customer records, but that figure does not show how many of the matches connect to the correct person.
A high match rate is only useful when the matches are accurate.
Businesses making major budget decisions can also use controlled tests. One group of customers is shown the campaign while another similar group is held back from seeing it. Their results can then be compared.
The Interactive Advertising Bureau says controlled experiments and holdout groups provide strong evidence when a company wants to establish whether advertising caused additional sales.
In the new study, tests that kept people in their original assigned groups were more resistant to missing advertising records than models that tried to estimate performance from observed behaviour.
Even controlled tests can still be weakened when the original customer matching is incorrect, so no method removes the need for reliable data.
The proposed solution also needs scrutiny
LiveRamp recommends stronger customer identity systems and controlled data-sharing environments, often known as data clean rooms.
LiveRamp has a commercial interest in this conclusion. The company sells identity resolution and data collaboration services, and its staff and resources were used in the research, according to the report announcement.
That does not make the simulation invalid, but businesses should not treat its headline figure as an independent industry-wide benchmark.
Better customer matching also creates privacy questions.
The US Federal Trade Commission has warned that data clean rooms are not automatically privacy-preserving. Their safety depends on how customer data is collected, connected, analysed and shared.
Companies therefore need to balance accurate advertising measurement with responsible handling of customer information.
The wider message from the study is useful even for businesses that never purchase an identity-matching product.
A marketing dashboard is built from records, customer links and assumptions. Before moving money or cancelling a campaign, a business needs to check whether those foundations are sound.
