A bank’s test of AI-generated customer models produced higher response rates for two kinds of financial offers, but the comparison does not isolate the effect of AI alone.
The research involved approximately 330,000 customers and compared the bank’s usual messaging with a process that used customer digital twins to help personalize offers. A digital twin in this setting is a model intended to approximate how a customer might respond, not a real person answering a survey.
The September 2026 working paper was produced by researchers at consultancy Oliver Wyman and Columbia Business School. It is preliminary research, not a guarantee that other businesses will get the same result.
What the bank compared
The researchers report two field experiments involving credit card and payroll transfer offers. The bank’s existing models helped select customers, while the new process used models built from relevant account and spending information to help choose how offers were presented.
Compared with the existing approach, the paper reports a 6% relative increase in credit card conversions and a 25% relative increase in payroll-transfer conversions. These are relative changes, not percentage-point increases.
The credit card result was only marginally statistically significant, according to Columbia’s account of the work. The personalized group also received more varied messages, so the comparison cannot show how much of the difference came from the twins rather than message variety.
The first 12 authors work for Oliver Wyman, which funded the research. That interest should be considered alongside the study’s methods and limitations.
Why a simulated customer is not a customer
A model can generate a plausible answer without accurately reflecting a person’s preferences. In separate Columbia research, twins built from extensive survey answers only modestly outperformed a simpler demographic baseline across the outcomes studied.
That comparison differs from the bank test. Predicting broad survey responses and choosing among actual marketing messages are different tasks, built on different information. Results from one should not be used to promise performance in the other.
Questions to ask before using the approach
Start with a specific decision and compare any new model with a simpler method. For example, a retailer might test whether a recommendation produces more completed purchases than its current system, not merely whether the generated explanation sounds convincing.
Use a forward test with a meaningful comparison group where practical. Measure outcomes relevant to customers and the business, including returns or complaints where appropriate, as well as immediate clicks. Small samples and multiple simultaneous changes make it harder to interpret results.
Customer information also requires appropriate privacy and security controls. Businesses need a valid basis for using data and should not assume a model’s output is correct merely because it sounds personal.
Our report on customer feedback and product design examines a different reason to check favorable signals against actual performance.
The bank research suggests a possible use for digital twins in one marketing workflow. Its most useful lesson for other companies is to test the whole system against a realistic alternative and examine what customers actually do.
