Tw-StAR at SemEval-2018 Task 1: Preprocessing Impact on Multi-label Emotion Classification

Abstract

In this paper, we describe our contribution in SemEval-2018 contest. We tackled task 1 "Affect in Tweets", subtask E-c "Detecting Emotions (multi-label classification)". A multilabel classification system Tw-StAR was developed to recognize the emotions embedded in Arabic, English and Spanish tweets. To handle the multi-label classification problem via traditional classifiers, we employed the binary relevance transformation strategy while a TF-IDF scheme was used to generate the tweets' features. We investigated using single and combinations of several preprocessing tasks to further improve the performance. The results showed that specific combinations of preprocessing tasks could significantly improve the evaluation measures. This has been later emphasized by the official results as our system ranked 3 rd for both Arabic and Spanish datasets and 14 th for the English dataset.

Hybrid Feature Model for Emotion Recognition in Arabic Text

“…Moreover, the HEF+DF model outperformed Badaro et al [2] by 2.3%, 1.3%, and 4.1% on Jaccard accuracy, F micro , and F macro , respectively. It also outperformed Mulki et al [3] by 4.7%, 3.4%, and 5.6% on Jaccard accuracy, F micro , and F macro , respectively. Finally, it outperformed the Abdullah and Shaikh [4] model by 6.6%, 5.9%, and 5.5% on Jaccard accuracy, F micro , and F macro , respectively.…”

“…The number of participants in the SemEval-2018 competition for emotion recognition in Arabic compared to the number of English participants was low. Of the eleven participants, only five achieved results higher than the baseline, and of those five, only Badaro et al [2], Mulki et al [3], and Abdullah and Shaikh [4] submitted a paper describing their systems.…”

“…Linear SVC with L1 outperformed the other learning models. Mulki et al [3] formulated multi-label emotion recognition as a binary classification problem and tested different preprocessing steps. The preprocessing pipeline used in their best results replaced emoji with emotion tags and performed stemming and stop-word removal.…”

The Evolution of Language Models Applied to Emotion Analysis of Arabic Tweets

“…For the Arabic dataset, 13 teams competed, the teams that achieved the highest results mostly used deep learning approaches such as convolutional neural networks (CNN), long short-term memory (LSTM), and bidirectional LSTM (Bi-LSTM), and some used traditional machine learning approaches such as support vector machine (SVM) with features such as sentiment and emotion lexicons or word embeddings. Out of the 13 participants, only Badaro et al [30], Mulki et al [31], and Abdullah and Shaikh [32] submitted a paper describing their systems. Badaro et al [30] used traditional machine learning (SVM, ridge classification (RC), random forests (RF), and an ensemble of the three) with features such as n-grams, affect lexicons, sentiment lexicons, and word embeddings from AraVec and fastText.…”

“…AraVec embeddings outperformed the other features. Mulki et al [31] also used SVM with TF-IDF to represent features, their main approach was in testing different preprocessing steps. Abdullah and Shaikh [32] used deep learning techniques by utilizing AraVec word embeddings and feeding them into four dense neural networks (DNNs).…”