Active Learning for Segment-based Emotion Analysis System

Abstract

Emotion detection is one of the challenging tasks in the fine-grained sentiment analysis fields. Traditional approaches focus on supervised learning with features based on surface word forms, without linguistic structure data, existing sparse problems and cannot mine implicit information including collocation and semantic prosody,which result in poor performance, especially for identifying implicit emotion expression. To improve the performance of emotion detection system,this paper deals with assigning Ekman's six basic emotion tags to sentences in Chinese written text. Baseline supervised system is developed based on segment based unit, and an active learning strategy using unlabeled data is presentedto improve the performance of baseline supervised system. Experimental evaluation shows that our active learning procedure helps to improve the classification accuracy.Key words:Emotion Analysis,Segment-based Unit,Active Learning 1.INTRODUCTIONEmotion analysis aims to identify the emotion categories (e.g., happy, angry, sadness and surprise) of a given written text. For example, the Chinese sentence -xiao meimei yi ting ku le qi lai, bian ku bian shuo: -nan dao ni men jiu bu neng de jiu ma? ‖‖ (The little sister wept and said, "Can you not be set free?"), which emotion category is -sadness‖. Labeling the written text with their emotion categories has drawn the attention of the natural language processingresearch community due to its many applications.Recentlythe most widely used methods for emotion analysis are supervised learning algorithms, such as Naive Bayes (NB), Support Vector Machine (SVM)and Condition Random Fields (CRF) etc. Supervised-based models (Sunita Sarawagi, 2004;Karl Moritz Hermann,2013) usually require hand-labeled data in order to achieve high-quality performance. However, the scarcity of hand-labeled data seriously compromises the development of automatic emotion analysis systems (EAS). One way of improving the performance of EAS (Richard Socher,2012) is to use extra annotation, typically unlabeled data, which are nowadays pervasive in digital format and relatively easy and inexpensive to collect (e.g., documents off the web).Unfortunately, the application of either post-editing or EAS to unlabeled data with massive data volumes is still too expensive, simply because manual supervision of all instances requires huge amounts of manpower. For such massive data streams the need of employing active learning (AL) is compelling. AL techniques (Jeff Mitchell,2008) for EAS selectively ask a human annotator to supervise a small portion of the incoming sentences. Sentences are selected so that EAS models estimated from them classify emotion category as accurately as possible.Thispaper describes an active learning algorithm for EAS. The modelhas been extended to incorporate segment-based emotion analysis as presented here. As the unlabeled data size becomes huge ina practical setting, and the active learning considers the amount of manual annotation for each sentence, the sentence sampling t...