NTUA-SLP at SemEval-2018 Task 1: Predicting Affective Content in Tweets with Deep Attentive RNNs and Transfer Learning

Abstract

In this paper we present deep-learning models that submitted to the SemEval-2018 Task 1 competition: "Affect in Tweets". We participated in all subtasks for English tweets. We propose a Bi-LSTM architecture equipped with a multi-layer self attention mechanism. The attention mechanism improves the model performance and allows us to identify salient words in tweets, as well as gain insight into the models making them more interpretable. Our model utilizes a set of word2vec word embeddings trained on a large collection of 550 million Twitter messages, augmented by a set of word affective features. Due to the limited amount of task-specific training data, we opted for a transfer learning approach by pretraining the Bi-LSTMs on the dataset of Semeval 2017, Task 4A. The proposed approach ranked 1 st in Subtask E "Multi-Label Emotion Classification", 2 nd in Subtask A "Emotion Intensity Regression" and achieved competitive results in other subtasks.

Detecting Emotions in English and Arabic Tweets

“…According to recent research by Young et al [4], deep learning methods are currently the state-of-the-art results in many domains. This trend was confirmed by the number of entries in the recent SemEval-2018 competition [5], where the winning entries in the English and Spanish [6,7] multi-classification competition all made use of neural networks. Indeed, many of the top-performing entries made use of a special kind of neural network known as "long short-term memory", either in conjunction with attention mechanisms [8,9], word embeddings [10,11] or convolutional neural networks [12,13].…”

“…Emotion classification tasks have traditionally been tackled by obtaining a large corpus, constructing feature sets, pre-processing in sophisticated ways and then making use of any number of black box training algorithms [14,15]. Indeed, this approach is still prevalent currently [6,7,16]. We demonstrate that with a small-sized corpus and without using any black box algorithms, our results are comparable with systems that have been trained on millions (and sometimes billions) of tweets.…”

“…We believe that, in this context, to be outperformed by only 0.005 serves to highlight the effectiveness of our approach. In the English competition, the entries that outperformed us were either based on neural networks [6,[9][10][11]13,33] or word embeddings [7,12,34], with a number of these also making use of external lexicons. The problem with using lexicons is that lexicons, stop-word lists and resources of this type are not always available for all languages.…”

A Deep Learning-Based Approach for Multi-Label Emotion Classification in Tweets

“…Baziotis et al [33], the winner of the multi-label emotion classification task of SemEval-2018 Task1: Affect in Tweets, developed a bidirectional Long Short-Term Memory (LSTM) with a deep attention mechanism. They trained a word2vec model with 800,000 words derived from a dataset of 550 million tweets.…”

“…• NTUA-SLP: the system submitted by the winner team of the SemEval-2018 Task1:E-cchallenge[33].• TCS: the system submitted by the second place winner[34]. • PlusEmo2Vec: the system submitted by the third place winner[35].•Transformer: a deep learning system based on large pre-trained language models developed by the NVIDIA AI lab[39].…”