AI-generated customer responses produced materially different pricing implications from human responses across four product studies, even when some simulated answers looked economically plausible, researchers report.
In one comparison, synthetic respondents’ average willingness to pay for a chocolate product was about 86% higher than the human benchmark. For a glass bottle, the average was only about 3% higher, but the distributions still differed and led to different modeled pricing outcomes.
The Journal of Marketing Analytics study by Sebastian Oetzel and Tobias Maiberger compared simulated answers generated with OpenAI GPT models against human valuations for four consumer products. The figures describe those tests, not an average error across AI systems or markets.
Why a plausible average can mislead
Willingness to pay is the maximum amount a customer would pay for a product. Businesses use estimates of it to assess how demand might change at different prices.
The human participants gave a maximum price under a mechanism in which a randomly drawn price could determine whether they bought the item. The researchers then generated synthetic respondents matched to participant characteristics and asked models for comparable valuations.
For the two products described as more pleasure-oriented, synthetic responses overstated average willingness to pay. For the two more practical products, results varied: the glass-bottle average was close, while the estimate for earplugs was lower than the human benchmark.
Even a close average can conceal a different spread of answers. If an AI estimate puts too many people at the high end of the price range, a model could recommend a price or forecast profit that does not match the human data.
The method also affected the answers
Some original prompts included a numerical price range that human participants had not been given. In additional tests, removing that information changed the AI estimates substantially, but it did not improve the match consistently across products.
The studies used GPT-family models, with different versions across the product tests. The research therefore cannot isolate product type from every model or prompt difference, or establish how other AI systems would perform.
How businesses can use the finding
Synthetic responses may help a team explore questions or prepare a research plan. Before using them to set a price, the researchers recommend checking them against human data gathered under conditions relevant to the decision.
A practical validation compares more than the average: examine the range of valuations, predicted demand at several prices and whether the recommended price holds up in an appropriately designed customer test. Include costs and uncertainty when projecting profit.
Human research also has sampling and design limitations. The useful comparison is not perfect people against imperfect software, but whether a method provides enough reliable evidence for the particular commercial decision.