An advertising platform can deliver campaigns to people likely to buy while missing others whose decisions are more likely to be changed by an advertisement, according to a randomized field experiment.
The finding concerns a common challenge for advertisers: a purchase recorded after an ad is not necessarily a purchase caused by that ad. Some customers would have bought the product anyway.
The study, published in Management Science, examined a display retargeting campaign involving 208,538 consumers. It found evidence that the platform’s incentives and the advertiser’s interests did not always point toward the same audience.
The results describe one campaign and a specific payment arrangement. They are not proof that every advertising platform or cost-per-purchase contract performs the same way.
Purchase likelihood and advertising effect
Programmatic advertising uses automated systems to bid for advertising space. In a retargeting campaign, the audience has previously interacted with the advertiser, such as by visiting its website.
An advertiser may pay a platform according to purchases attributed to a campaign. Under such an arrangement, the platform has a reason to find people likely to purchase after an ad is shown.
Yet a customer’s baseline likelihood of buying and the effect of the advertisement are separate questions. Someone who was already planning to buy may purchase after seeing an ad even if the ad made no difference. Another person might buy only because the message introduced a product or resolved a concern.
Researchers Thomas W. Frick, Rodrigo Belo and Rahul Telang used a randomized experiment to estimate the effect of advertisements on purchase probability and compare it with the platform’s targeting.
What the field experiment found
The campaign assigned consumers to a group eligible to see the advertiser’s display ads or a control group shown public-service advertisements. The researchers tracked subsequent purchases and website visits.
On average, the advertising group was more likely to return to the website and more likely to purchase. But the researchers found that the platform tended to target people with higher baseline purchase probabilities, rather than people who were necessarily more responsive to the ads.
The paper models how the difference can affect the advertiser’s return. The authors also examine ways an advertiser could restrict targeting using estimates of both baseline purchase likelihood and responsiveness to advertising.
Those proposed methods rely on data and assumptions. They should not be treated as a guaranteed improvement for a different business or platform.
Measuring incremental sales
Incremental sales are purchases that occur because of an action and would not have occurred otherwise. They cannot be identified simply by counting everyone who purchased after seeing a campaign.
Where feasible, an advertiser can compare randomly assigned audiences that do and do not receive a campaign. The difference in outcomes provides evidence about the campaign’s effect for the population and period tested.
Our earlier coverage of retail app advertising also examined why measures of activity and purchases answer different questions. An app installation, a click and an additional profitable order are not interchangeable results.
To assess a campaign, a business can compare estimated additional gross profit with the full cost of serving ads and fulfilling the extra orders. An increase in purchases may still be unprofitable if advertising costs are too high.
Questions to ask a platform
Advertisers can ask how purchases are attributed, how long the attribution window lasts, and whether reported results include customers who might have bought without seeing an ad.
They can also ask whether a holdout test is available and how the platform handles customers who see advertising across multiple devices. Tests require enough observations and a suitable comparison group; small differences can be difficult to distinguish from chance.
The study’s broader lesson is to align the measurement with the business decision. A campaign can reach people likely to buy and still create fewer additional profitable purchases than its reported conversions imply.