Researchers analyzing 2,226 Meta advertising experiments say results from a subset of randomized tests can help predict the additional conversions generated by other campaigns.
The proposed method addresses a familiar problem for marketers: a customer may see or click an advertisement and then buy something they would have bought anyway. Counting the sale after the click does not establish that the advertisement caused it.
Brett Gordon, Robert Moakler and Florian Zettelmeyer describe their approach in a National Bureau of Economic Research working paper. The findings concern campaigns in their dataset and should not be treated as proof that the method will perform equally well on every platform or business.
Measuring the sales an ad adds
Advertising incrementality is the difference between the results of showing an advertisement and the results expected without it. A randomized controlled trial estimates that difference by assigning comparable groups to different conditions, such as exposure to an ad or a control.
Randomized tests can be informative, but running enough of them across campaigns can be difficult and costly. The researchers propose a method called predicted incrementality by experimentation, or PIE, which learns from a sample of experiments and uses campaign characteristics to predict effects for campaigns without their own test.
According to the working paper, the prediction can use campaign-level information available after a campaign ends, including exposure rates and last-click conversions. The authors emphasize that the randomized tests supply the causal estimates used to train the model. A post-campaign correlation by itself would not establish causation.
What the comparison found
For incremental conversions per dollar, the researchers report an out-of-sample R-squared of 0.88 for their model, compared with 0.19 for seven-day last-click attribution. Out-of-sample testing assesses predictions on data not used to fit the model; R-squared is a measure of how closely predictions account for variation in the outcome.
Last-click attribution assigns credit to an ad interaction near the purchase. It can be easy to calculate, but it does not isolate customers who would have purchased without that ad. Our guide to measuring incremental marketing sales explains how a holdout group provides a comparison.
In the paper’s decision framework, PIE disagreed with decisions based on randomized tests in 8% to 12% of campaigns, compared with 12% to 20% for last-click attribution. Those ranges depend on the authors’ framework and are not guarantees for an individual advertiser.
Moakler is a Meta employee and shareholder, and Gordon and Zettelmeyer previously worked part time for Facebook. The paper discloses those relationships. It is preliminary research, and the evidence comes from Meta experiments rather than a representative sample of all advertising channels.
Prediction still needs good experiments
A business cannot tell from the figures alone whether it should increase a particular advertising budget. It would need a relevant set of reliable tests, comparable campaigns and a way to check predictions as markets, targeting and customer behavior change.
Even a useful prediction can be wrong for a particular campaign. Marketers also need to decide what outcome matters: an additional purchase, for example, is not the same as additional profit after product, advertising and service costs.
The paper’s proposed contribution is to use experimental evidence beyond the campaigns in which it was collected. Its performance outside the studied setting remains an empirical question for advertisers considering the approach.
