Assessing State-of-the-Art Sentiment Models on State-of-the-Art Sentiment Datasets

Abstract

There has been a good amount of progress in sentiment analysis over the past 10 years, including the proposal of new methods and the creation of benchmark datasets. In some papers, however, there is a tendency to compare models only on one or two datasets, either because of time restraints or because the model is tailored to a specific task. Accordingly, it is hard to understand how well a certain model generalizes across different tasks and datasets. In this paper, we contribute to this situation by comparing several models on six different benchmarks, which belong to different domains and additionally have different levels of granularity (binary, 3-class, 4-class and 5-class). We show that BiLSTMs perform well across datasets and that both LSTMs and Bi-LSTMs are particularly good at fine-grained sentiment tasks (i. e., with more than two classes). Incorporating sentiment information into word embeddings during training gives good results for datasets that are lexically similar to the training data. With our experiments, we contribute to a better understanding of the performance of different model architectures on different data sets. Consequently, we detect novel state-of-the-art results on the SenTube datasets.

Self-Attention: A Better Building Block for Sentiment Analysis Neural Network Classifiers

“…To reduce implementation deviations from previous work, we use the codebase from [Barnes et al, 2017] and only replace the model and training process. We re-use the code for batch preprocessing and batch construction for all datasets, accuracy evaluation, as well as use the same word embeddings 2 .…”

“…Our goal is to test the effectiveness of the main building blocks. We compare directly the results of two proposed architectures, 1-Layer-SSAN and 2-Layer-SSAN, to the LSTM, BiLSTM, and CNN architectures from [Barnes et al, 2017].…”

“…We take 5 classifiers from [Barnes et al, 2017] and use their published results as baselines. Two of the methods are based on logistic regression, Bow and Ave, and 3 are neural network based, LSTM, BiLSTM, and CNN.…”

“…Test accuracy averages and standard deviations (in brackets) of 5 runs. The baseline results are taken from [Barnes et al, 2017]; the self-attention models results are ours. Best model for each dataset is given in bold .…”

A Computational Linguistic Study of Personal Recovery in Bipolar Disorder

“…Furthermore, emotion and sentiment analysis constitute useful tools to investigate the emotions involved in talking about recovery and identify factors that facilitate or hinder it. There are many annotated datasets to train supervised classifiers (Bostan and Klinger, 2018;Barnes et al, 2017) for these actively researched NLP tasks. Machine learning methods were found to usually outperform rule-based approaches based on look-ups in dictionaries such as LIWC.…”