The Impact of Sentiment Analysis Output on Decision Outcomes: An Empirical Evaluation

Abstract

Abstract:User-generated online content serves as a source of product-and service-related information that reduces the uncertainty in consumer decision making, yet the abundance of such content makes it prohibitively costly to use all relevant information. Dealing with this (big data) problem requires a consumer to decide what subset of information to focus on. Peer-generated star ratings are excellent tools for one to decide what subset of information to focus on as they indicate a review's "tone". However, star ratings are not available for all user-generated content and not detailed enough in other cases. Sentiment analysis, a text-analytic technique that automatically detects the polarity of text, provides sentiment scores that are comparable to, and potentially more refined than, star ratings. Despite its popularity as an active topic in analytics research, sentiment analysis outcomes have not been evaluated through rigorous user studies. We fill that gap by investigating the impact of sentiment scores on purchase decisions through a controlled experiment using 100 participants. The results suggest that, consistent with the effort-accuracy trade off and effort-minimization concepts, sentiment scores on review documents improve the efficiency (speed) of purchase decisions without significantly affecting decision effectiveness (confidence).

An improved evidence-based aggregation method for sentiment analysis

“…SA can be used as an offer to purchase or not to purchase a particular product [2], as well as a consultant for product makers to extract the customers’ desirable features and improve the quality of products and services [3]. Although opinion mining and SA are referred to the same field of study by some authors [4], by definition, opinion mining is only the detection of positivity, negativity or neutrality of reviews, but in SA, each word containing an opinion is given a weight based on the subject of the text and the polarity of that word [1].…”

User Satisfaction with Wearables

“…I measured disconfirmation effects via a polarity score calculated with a sentiment analysis algorithm for each review. These sentiment scores offer more granular and refined evaluations than star ratings (Lak & Turetken, 2017). Since polarity scores evaluate performance with a continuous variable that can take positive or negative values in a more nuanced way, I used these scores to operationalize dis/confirmation effects in the reviews.…”

Untitled

“…Online reviews, a specific subset of electronic word of mouth (eWOM), have become an integral component of consumers' information-seeking behavior and subsequent purchase decision making and, therefore, a powerful marketing tool (Bickart & Schindler, 2001;Gruzd, 2013;Kumar & Benbasat, 2006;Lak & Turetken, 2017;Zhang, Craciun, & Shin, 2010). Online reviews allow consumers to reduce the uncertainty and risk associated with purchase decisions and, therefore, increase their level of confidence during such decision making processes (Pitta & Fowler, 2005).…”

“…For instance, Cheung, Sia, and Kuan (2012) examined review consistency and sidedness. Other review characteristics that have received a lot of attention include review valence (commonly referred to as sentiment) (Benedicktus & Andrews, 2006;Gruzd, 2013;Lak & Turetken, 2017;Stauss, 2000), review accuracy (Nelson, Todd, & Wisom, 2005;Yoo, Kim, & Sanders, 2015;Cheung, Lee, & Rabjohn, 2008), and review currency (Kahana, Howard, Zaromb, & Wingfield, 2002;Wathen & Burkell, 2002). Furthermore, beyond a review's credibility, some research has examined reviewers' perceived credibility (i.e., how much of an expert reviewers are with respect to the particular product or service) (Cheung et al, 2008;Cheung, Luo, Sia, & Chen, 2009) as an additional antecedent that affects the information processing that occurs when consumers read online reviews.…”