featured-image-standard-abstract-lines-dots-nodes-connecting

Using machine learning to yield useful market insight

Written by Joseph Nordqvist

Published: 04:36, February 10, 2019

Gauging consumer needs is essential in marketing. Focus groups, interviews and surveys are currently the most common means of gathering this data. But the process can be time-consuming and expensive.

The advent of machine learning technology and artificial intelligence (AI) has sparked interest in using the technology to yield valuable insights into consumer wants.

Researchers at MIT devised a method of efficiently identifying customer needs from user-generated content (UCG) with machine learning, according to a study published in Marketing Science.

UCG is content that has been created and put out there by unpaid contributors, including posts on online platforms such as social media and wikis.

One of the biggest challenges with using UCG for consumer insights is that a lot of it is noninformative or repetitive.

Authors of the study, Artem Timoshenko and John R. Hauser, used a neural network that filtered out noninformative content and clustered dense sentence embeddings to avoid sampling repetitive content.

The results showed that machine-learning methods improve efficiency of identifying customer needs from UGC. In addition, the authors said that UGC proved to be “as valuable as a source” of customer needs for product development, “likely more valuable”, compared with “conventional methods.”

“As more and more people turn to the digital marketplace to research products, share their opinions, and exchange product experiences, large amounts of UGC data is available quickly and at a low incremental cost to companies,” said Timoshenko.

“In many brand categories, UGC is extensive. For example, there are more than 300,000 reviews on health and personal care products on Amazon alone. If UGC can be mined for customer needs, it has the potential to identify customer needs better than direct customer interviews.”

Hauser commented: 

“In the end, we found that UGC does at least as well as traditional methods based on a representative set of customers.”

“We were able to process large amounts of data and narrow it to manageable samples for manual review. The manual review remains an important final part of the process, since professional analysts are best able to judge the context-dependent nature of customer needs.”

Citation

“Identifying Customer Needs from User-Generated Content” Artem Timoshenko, John R. Hauser. 30 Jan 2019. Marketing Science. https://doi.org/10.1287/mksc.2018.1123

Joseph Nordqvist Avatar

Other News

Why the weather can affect ratings of cold-weather gear

Nov 15, 2026

Seeing a purchase as part of a collection can change the shopping experience

Nov 15, 2026

Retail app ads may influence what customers do after installation

Nov 15, 2026

A bank tested AI customer twins for marketing, with mixed lessons for businesses

Nov 15, 2026

What makes a product vintage, and why the distinction matters to retailers

Nov 15, 2026

EU business social media use rises, but a gap remains by company size

Nov 15, 2026

Why startups may benefit from testing locally before expanding abroad

Nov 14, 2026

Novel product ideas can be harder to try but more rewarding afterward

Nov 14, 2026

What is in an online cart can influence whether shoppers check out

Nov 14, 2026

Subscription plans can miss customers who would save money, Lyft research finds

Nov 14, 2026

Workers with worse shifts than their colleagues were more likely to leave, restaurant study finds

Nov 14, 2026

More product options do not always reach more customers, cosmetics study finds

Nov 14, 2026

EU household spending grew in 2025 as communication purchases led gains

Nov 13, 2026

Ad experiments can help predict which campaigns drive additional sales

Nov 13, 2026

Browser choice screens lifted Firefox usage in Europe, researchers find

Nov 13, 2026

Online product rankings can cut shipping costs without reducing sales, experiment finds

Nov 13, 2026

Large companies lead EU service exports, but smaller firms dominate in some markets

Nov 13, 2026

Cart reminder timing is linked to different shopper responses

Nov 12, 2026

AI labels changed how consumers judged food images in three experiments

Nov 12, 2026

Shared AI tools were linked to higher sales for small online sellers

Nov 12, 2026