Multi-Label Classification of Parrott’s Emotions

Abstract

Mining for latent emotions embedded in tweets can offer clues about users' affective state on a broad range of topics ranging from their mental health to political opinions. This paper presents a multi-class supervised learning approach to group tweets into six emotions (joy, sadness, anger, fear, love, and surprise) defined according to the Parrott's framework. After extensive pre-processing, linguistic and metadata features extracted from a corpus of tweets are used to train popular machine learning classifiers. The performance of these classifiers is evaluated using accuracy, sensitivity, and specificity computed based on a multi-class confusion matrix approach. Our framework can detect common emotions of joy and sadness with excellent accuracy (> 90%), anger and fear with moderate accuracy (75% − 85%), and love and surprise with lower accuracy (50% − 60%). Overall, the accuracy of our framework still outperforms that of contemporary approaches for all the six emotions. Further analysis of an example multi-class confusion matrix indicates that lower accuracy values for love and surprise may arise because love is often confused with joy, whereas surprise is mixed up with the positive emotion of joy and the negative emotion of fear. Moreover, this confusion could be attributed to an under-representation of these emotions in the data. This highlights the need for building high-quality, balanced benchmark data sets for training multi-label emotion classifiers.