A controlled advertising experiment compares customers assigned to receive a campaign with customers assigned to a group where it is withheld. The difference in their outcomes helps estimate how much business the campaign adds. A report that credits an advertisement with a sale answers a different question.
For a business deciding where to spend its next advertising dollar, knowing which channel appeared before a purchase is only a starting point. Some of those customers might have bought without the campaign. Others may need several contacts before making a decision.
The task is to connect marketing activity with additional customers, revenue and money left after serving them. That requires a consistent sales record, a credible comparison and enough time for purchases to occur.
What attribution can tell you
Marketing attribution means assigning credit for a purchase or another action to advertisements and other contacts along the customer’s path. A last-click model gives the credit to the final eligible contact. Other models distribute it across several contacts.
Google’s attribution documentation describes rules and data-driven models for making that allocation. These reports can help identify the paths associated with purchases and compare campaign performance under a consistent method.
An allocation of credit does not, by itself, establish how many purchases would disappear if the advertising stopped. Even sophisticated models depend on the information available and the assumptions used. Some incorporate experimental evidence, but their estimates still need to be understood before becoming a budget rule.
A 2019 study in Marketing Science compared several observational approaches with 15 US advertising experiments at Facebook. The dataset contained 500 million user-experiment observations, not necessarily that many different people.
Brett Gordon, Florian Zettelmeyer, Neha Bhargava and Dan Chapsky found that the observational methods often failed to reproduce the experimental effects, despite extensive information about customers. The research concerns those campaigns and methods; it does not establish that every modern attribution system is inaccurate.
Businesses should also avoid adding together the sales claimed by separate platforms without checking for overlap. A customer can encounter several campaigns before placing one order. The order should appear once in the business’s sales records, even if several reporting systems claim a connection.
Compare groups that had the same chance to buy
Incrementality is the additional outcome caused by an activity. A holdout test estimates it by keeping a randomly selected comparison group outside a campaign while the other group remains eligible to receive it.
Random assignment reduces the risk of comparing people who already intended to buy with people who had little interest. The analysis should follow the assigned groups, including eligible customers who never actually saw an advertisement.
Google’s Conversion Lift service is one example of this approach. It offers user-based and geographical studies. Availability and suitability depend on the account and campaign, so a business should check the current requirements before planning a test.
Choose the outcome before starting. A retailer might count completed, non-refunded orders. A business selling services might count qualified inquiries, then follow them through to signed work. The same definition and collection method must apply to both groups.
Compare rates or properly adjusted totals when the groups differ in size. Report the uncertainty around the result, rather than treating a small numerical lead as proof that the campaign worked.
Keep the observation period long enough for the normal buying process. Google’s study guidance specifically addresses conversion lag, the time between advertising contact and a later action. A test ending before customers normally purchase can miss part of the response.
Pricing, stock availability and other campaigns can interfere with comparisons. Switching advertising off during a quiet week and comparing it with a busy holiday week gives a weak basis for claiming a causal effect. Randomized tests or carefully designed geographical comparisons offer a more credible starting point.
Work out what the extra sales leave behind
Return on advertising spend, often shortened to ROAS, divides the revenue credited to advertising by the advertising cost. Incremental ROAS uses the extra revenue estimated through a test. Both are revenue measures.
Neither tells the owner how much remains after paying to provide the product or service. Product costs, delivery, payment charges, returns and extra sales commissions may absorb part of the additional revenue.
Contribution margin is revenue less variable costs, the costs that increase with the activity being measured. To assess a campaign, estimate the contribution from the additional business, then deduct the campaign’s additional costs. Avoid counting the same cost twice.
Existing fixed expenses may stay unchanged during a small campaign. A larger rollout could require another employee, extra premises or new equipment. Those commitments belong in the expansion decision even if the initial experiment did not incur them.
Repeat purchases can improve the economics, but expected future orders are not cash already earned. Track the customers acquired in the same period as a group, recording their subsequent revenue, service costs and retention. Keep observed results separate from projections.
Make the test fit the size of the business
A small advertiser may have too few completed sales to detect a modest difference reliably. An inconclusive result means the test could not distinguish the effect clearly enough. It does not prove that advertising has no effect.
A narrower question can be easier to answer. A business might test an additional email campaign among consenting subscribers, for example, while keeping prices and the normal communications schedule consistent. The result would apply to that campaign and audience, not automatically to every marketing channel.
Before starting, record the audience, outcome, comparison method, observation period and spending limit. Decide what result would justify continuing, reducing or expanding the campaign. A platform report and a spreadsheet of actual orders should use the same periods and treatment of cancellations.
If a result remains uncertain, retain the range of plausible outcomes in the budget discussion. Expansion can then proceed in stages, with further measurement as spending increases. A campaign that works at its initial scale may reach a different mix of customers when the budget grows.
