Researchers have found that machine learning can really improve the process of identifying consumer needs. Consumer brands globally have long used old-fashioned focus groups. They have also used surveys and interviews to gauge consumer needs, wants, and desires. They have used them as part of the processes that include marketing, sales, and product development.
With the emergence of machine learning and AI, there has been growing interest in their ability to harness these solutions to save money and time. Marketers and other business people have also wondered whether AI might yield more reliable consumer insights.
Identifying consumer needs – we still need humans
Machine learning can help in the analysis of user-generated content or UGC. UGC involves gathering data from blogs, social media, and online reviews. In other words, data from material that provides insights on consumer attitudes, preferences, and needs.
However, UGC is difficult to process because of the sheer volume of data. Marketers, therefore, wonder about its value. Even though the data is accessible, we still need humans to identify consumer insights. In other words, we need people to analyze the data. This is difficult to do on a large scale.
Artem Timoshenko and John R. Hauser decided to address this problem through research. They wanted to find the most efficient ways of using UGC to identify customer needs. They also sought to find the most cost-efficient and accurate ways of doing this.
The researchers wrote about their work in the journal Marketing Science (citation below).
Identifying consumer needs – machine learning improves process
The authors found that machine learning could improve the process of identifying consumer needs. It also reduced research time significantly, helping avoid consumer marketing brand delays and bringing products to market more promptly.
Artem Timoshenko, a Ph.D. Candidate in Marketing at the MIT Sloan School of Management, said:
“As more and more people turn to the digital marketplace to research products, share their opinions, and exchange product experiences, large amounts of UGC data is available quickly and at a low incremental cost to companies. In many brand categories, UGC is extensive.”
“For example, there are more than 300,000 reviews on health and personal care products on Amazon alone. If UGC can be mined for customer needs, it has the potential to identify customer needs better than direct customer interviews.”
UGC data has another advantage – it is continuously updated. This enables businesses to remain current with their understandings of consumer needs. Also, unlike customer interviews, researchers can use UGC data to explore new insights further.
Identifying consumer needs with a machine-learning hybrid approach
The researchers created and evaluated a machine-learning hybrid approach to identify consumer needs from UGC data.
First, machine learning identified relevant content and filtered out what was irrelevant. The processed data was then analyzed by people to formulate customer needs from specific content.
John Hauser, the Kirin Professor of Marketing at MIT’s Sloan School of Management, said:
“In the end, we found that UGC does at least as well as traditional methods based on a representative set of customers. We were able to process large amounts of data and narrow it to manageable samples for manual review.”
“The manual review remains an important final part of the process since professional analysts are best able to judge the context-dependent nature of customer needs.”
Citation
“Identifying Customer Needs from User-Generated Content,” Artem Timoshenko and John R. Hauser. Marketing Science, Published Online:30 Jan 2019 DOI: https://doi.org/10.1287/mksc.2018.1123.
