Sentiment Scoring and Performance Metrics Examination of Various Supervised Classifiers

Abstract

Sentiment Analysis probes public opinion on user generated content on Web like blogs, social media or e-commerce websites. The results of Sentiment Analysis are getting much attention with marketers that they are able to evaluate the success of an advertising campaign or the attitude of people on a new product launch. Business owners and advertising companies are using Sentiment Analysis to start new business strategies and to identify opportunities for new product development. In this paper, with R programming, the tweets from Twitter about Samsung Galaxy mobile phone and Apple Iphone were retrieved from three countries namely USA, UK and India for creating the dataset. The collected tweets were classified into positive, negative and neutral sentiments. The machine learning classifier algorithms like Naïve Bayes, Support Vector Machine, Random Forest, Decision Tree, Artificial Neural Network, XGBoost with K Fold cross validation were applied on the dataset and the results were tabulated for comparing and estimating which classifier algorithm yields the best accuracy. Other performance metric values like F Score, Precision, Recall were also calculated for comparison of various classifier performances on Sentiment Analysis. It was found that XGBoost method combined with K Fold cross validation has produced the best accuracy in prediction. We have also applied SentiStrength algorithm to find out the intensity or the strength of positive and negative comments from each sentence. With the help of the results in hand, we were able to predict the brand of mobile phone that was preferred in each country.