Language Independent Sentiment Analysis with Sentiment-Specific Word Embeddings

Abstract

Data annotation is a critical step to train a text model but it is tedious, expensive and time-consuming. We present a language independent method to train a sentiment polarity model with limited amount of manuallylabeled data. Word embeddings such as Word2Vec are efficient at incorporating semantic and syntactic properties of words, yielding good results for document classification. However, these embeddings might map words with opposite polarities, to vectors close to each other. We train Sentiment Specific Word Embeddings (SSWE) on top of an unsupervised Word2Vec model, using either Recurrent Neural Networks (RNN) or Convolutional Neural Networks (CNN) on data auto-labeled as "Positive" or "Negative". For this task, we rely on the universality of emojis and emoticons to auto-label a large number of French tweets using a small set of positive and negative emojis and emoticons. Finally, we apply a transfer learning approach to refine the network weights with a small-size manuallylabeled training data set. Experiments are conducted to evaluate the performance of this approach on French sentiment classification using benchmark data sets from SemEval 2016 competition. We were able to achieve a performance improvement by using SSWE over Word2Vec. We also used a graph-based approach for label propagation to auto-generate a sentiment lexicon.

Design and validation of annotation schemas for aspect-based sentiment analysis in the tourism sector

“…It is therefore hardly surprising that most efforts have focused on the application of machine learning algorithms, which produce classifiers whose statistical model is learnt, typically, in a supervised fashion from a corpus of labeled examples, such as user reviews (Pang et al 2002;Turney 2002) or tweets (Nakov 2016). The techniques to improve performance of classifiers have progressively gained computational complexity over the years; at present, supervised methods such as Support Vector Machine (SVM), Bayesian Networks and Neural Networks, among others are commonplace, as are unsupervised learning techniques based on pre-trained word embeddings (such as Word2vec) or clustering techniques, e.g., Tang et al 2014;Kalchbrenner et al 2014;Kim 2014;Fu et al 2018;Saroufim et al 2018), with or without the aid of external lexical resources (sentiment dictionaries).…”

Data Mining for Cyberbullying and Harassment Detection in Arabic Texts

“…In [13], the authors suggested rating tweets in French using a dataset of positive and negative emojis and training them to include Sentiment Specific Word Embeddings (SSWE) on top of an unsupervised Word2Vec model. It updated the embedding through deep learning with bidirectional LSTM on the auto-labeled data.…”

Language Independent Sentiment Analysis

“…Another issue with pre-trained embedding is that it does not account for sentiment polarity of words and might map words with opposite polarities to vectors closer to each other in Euclidean space. A novel sentiment-specific word embedding is proposed for language-independent sentiment analysis, which shows an improvement over traditional pre-trained embeddings of word2vec [12].…”