Gauging consumer needs is essential in marketing. Focus groups, interviews and surveys are currently the most common means of gathering this data. But the process can be time-consuming and expensive.
The advent of machine learning technology and artificial intelligence (AI) has sparked interest in using the technology to yield valuable insights into consumer wants.
Researchers at MIT devised a method of efficiently identifying customer needs from user-generated content (UCG) with machine learning, according to a study published in Marketing Science.
UCG is content that has been created and put out there by unpaid contributors, including posts on online platforms such as social media and wikis.
One of the biggest challenges with using UCG for consumer insights is that a lot of it is noninformative or repetitive.
Authors of the study, Artem Timoshenko and John R. Hauser, used a neural network that filtered out noninformative content and clustered dense sentence embeddings to avoid sampling repetitive content.
The results showed that machine-learning methods improve efficiency of identifying customer needs from UGC. In addition, the authors said that UGC proved to be “as valuable as a source” of customer needs for product development, “likely more valuable”, compared with “conventional methods.”
“As more and more people turn to the digital marketplace to research products, share their opinions, and exchange product experiences, large amounts of UGC data is available quickly and at a low incremental cost to companies,” said Timoshenko.
“In many brand categories, UGC is extensive. For example, there are more than 300,000 reviews on health and personal care products on Amazon alone. If UGC can be mined for customer needs, it has the potential to identify customer needs better than direct customer interviews.”
Hauser commented:
“In the end, we found that UGC does at least as well as traditional methods based on a representative set of customers.”
“We were able to process large amounts of data and narrow it to manageable samples for manual review. The manual review remains an important final part of the process, since professional analysts are best able to judge the context-dependent nature of customer needs.”
Citation
“Identifying Customer Needs from User-Generated Content” Artem Timoshenko, John R. Hauser. 30 Jan 2019. Marketing Science. https://doi.org/10.1287/mksc.2018.1123
