Sentiment Extraction: Integrating Statistical Parsing, Semantic Analysis, and Common Sense Reasoning

Abstract

Much of the ongoing explosion of digital content is in the form of text. This content is a virtual gold-mine of information that can inform a range of social, governmental, and business decisions. For example, using content available on blogs and social networking sites businesses can find out what its customers are saying about their products and services. In the digital age where customer is king, the business value of ascertaining consumer sentiment cannot be overstated. People express sentiments in myriad ways. At times, they use simple, direct assertions, but most often they use sentences involving comparisons, conjunctions expressing multiple and possibly opposing sentiments about multiple features and entities,and pronominal references whose resolution requires discourse level context. Frequently people use abbreviations, slang, SMSese, idioms and metaphors. Understanding the latter also requires common sense reasoning. In this paper, we present iSEE, a fully implemented sentiment extraction engine, which makes use of statistical methods, classical NLU techniques, common sense reasoning, and probabilistic inference to extract entity and feature specific sentiment from complex sentences and dialog. Most of the components of iSEE are domain independent and the system can be generalized to new domains by simply adding domain relevant lexicons.

VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text

“…That is, when we fairly apply the rules to all lexicons and ML algorithms, we achieve better correlation coefficients (mean r increase of 5.2%) and better accuracies (mean F1 increase of 2.1%). Consistent with prior work (Agarwal, Xie, Vovsha, Rambow, & Passonneau, 2011;Davidov et al, 2010;Shastri, Parvathy, Kumar, Wesley, & Balakrishnan, 2010), we find that grammatical features (conventions of use for punctuation and capitalization) and consideration for degree modifiers like "very" or "extremely" prove to be useful cues for distinguishing differences in sentiment intensity. Other syntactical considerations identified via qualitative analysis (negation, degree modifiers, and contrastive conjunctions) also help make VADER successful, and is consistent with prior work (Agarwal et al, 2011;Ding, Liu, & Yu, 2008;Lu, Castellanos, Dayal, & Zhai, 2011;Socher et al, 2013).…”