Identifying Customer Needs from User-Generated Content

Abstract

Firms traditionally rely on interviews and focus groups to identify customer needs for marketing strategy and product development. User-generated content (UGC) is a promising alternative source for identifying customer needs. However, established methods are neither efficient nor effective for large UGC corpora because much content is non-informative or repetitive. We propose a machine-learning approach to facilitate qualitative analysis by selecting content for efficient review. We use a convolutional neural network to filter out non-informative content and cluster dense sentence embeddings to avoid sampling repetitive content. We further address two key questions: Are UGCbased customer needs comparable to interview-based customer needs? Do the machine-learning methods improve customer-need identification? These comparisons are enabled by a custom dataset of customer needs for oral care products identified by professional analysts using industry-standard experiential interviews. The analysts also coded 12,000 UGC sentences to identify which previously identified customer needs and/or new customer needs were articulated in each sentence. We show that (1) UGC is at least as valuable as a source of customer needs for product development, likely morevaluable, than conventional methods, and (2) machine-learning methods improve efficiency of identifying customer needs from UGC (unique customer needs per unit of professional services cost).

How online word‐of‐mouth impacts receivers

“…Ultimately, the effect of the marketing mix on receivers is dynamic: marketing mix variables increase sales and drive WOM (e.g., Berger & Schwartz, ; Fossen & Schweidel, ; Fossen & Schweidel, ; Kuksov & Xie, ; Lovett, Peres, & Shachar, ), which, in turn, affects product diffusion (e.g., Amblee & Bui, ; Chintagunta et al, ; Dellarocas Zhang & Awad, ; Gelper et al, ; Godes & Mayzlin, ; Park & Kim, ) and subsequent WOM (Park, Shin, & Xie, ). By monitoring WOM, sellers can assess consumers’ perceptions of product quality, changes in the importance of product attributes, and shifts in market position and competitiveness (e.g., Chen & Xie, ; Kwark, Chen, & Raghunathan, ; Netzer, Feldman, Goldenberg, & Fresko, ; Timoshenko & Hauser, ; Tirunillai & Tellis, ), and adjust the marketing mix accordingly.…”

A Machine Learning Approach to Customer Needs Analysis for Product Ecosystems

“…However, such methods demand a large number of resources from subject matter experts and thus are costly and time-consuming. Another challenge is how to effectively and efficiently analyze text data [8]. Whether the voice of customer (VoC) data are collected through traditional subjective methods or recent online product reviews, the large scale and unstructuredness of text data often make it costly to deal with.…”

“…Online User-Generated Data. Online user-generated data have proved to be a promising source to identify customer needs more efficiently and effectively than other data sources collected by traditional subjective methods [8]. First, such data are often large-scale and easy to obtain at a low cost.…”

Managing Marketing Decision-Making with Sentiment Analysis: An Evaluation of the Main Product Features Using Text Data Mining

“…For example, a positive sentence was, "Signal: Superb I have had many cel phones before including Nokia which I thing it has a great signal but HTC_TYTN_II has much better signal." This had a phrase sentiment score of 3.33 using Equation (4). Another example was a negative sentence: "Battery: Weak point do not expect your battery to last more than 24 h and much less if you use it heavily."…”

“…By writing blogs, participating in social media, or reviewing products online, internet users are constantly generating content. Consumer comments in online forums have proven to be a useful source for revealing consumer insights [3], and this user-generated content (UGC) represents a promising alternative source for potentially identifying customer needs [4]. Thus, mining this UGC and analyzing the sentiments of the comments expressed by consumers might be useful for companies.…”