A System for Affect Analysis of Utterances in Japanese Supported with Web Mining

Abstract

We propose a method for affect analysis of textual input in Japanese supported with Web mining. The method is based on a pragmatic reasoning that emotional states of a speaker are conveyed by emotional expressions used in emotive utterances. It means that if an emotive expression is used in a sentence in a context described as emotive, the emotion conveyed in the text is revealed by the used emotive expression. The system ML-Ask (Emotive Elements / Expressions Analysis System) is constructed on the basis of this idea. An evaluation of the system is performed in which two evaluation methods are compared. To choose the most objective evaluation method we compare the most popular method in the field and a method proposed by us. The proposed evaluation method was shown to be more objective and revealed the strong and weak points of the system in detail. In the evaluation experiment ML-Ask reached human level in recognizing the general emotiveness of an utterance (0.83 balanced F-score) and 63% of human level in recognizing the specific types of emotions. We support the system with a Web mining technique to improve the performance of emotional state types extraction. In the Web mining technique emotive associations are extracted from the Web using co-occurrences of emotive expressions with morphemes of causality. The Web mining technique improved the performance of the emotional states types extraction to 85% of human performance

Towards Computational Fronesis

“…Ptaszynski et al (2009a) already showed that ML-Ask and Shi's technique are compatible and can be used as complementary means to improve the emotion recognition task. However, these two methods are based on different assumptions.…”

ML-Ask: Open Source Affect Analysis Software for Textual Input in Japanese

“…Evaluations ML-Ask has been evaluated a number of times on different datasets and frameworks. In first evaluations, Ptaszynski et al [12,20,21] focused on evaluating the system on separate sentences. For example, in [20], there were 90 sentences (45 emotive and 45 non-emotive) annotated by authors of the sentences (first-person standpoint annotations).…”

“…On this dataset ML-Ask achieved 83% of balanced F-score for determining whether a sentence is emotive, 63% of human level of unanimity score for determining emotive value and 45% of balanced F-score for detecting particular emotion types. In [12] Ptaszynski et al added annotations of third-party annotators and performed additional evaluation from the third-person standpoint. The evaluation showed that ML-Ask achieves better performance when supported by additional Web-mining procedure (not included in the OpenSource version) for extracting emotive associations from the Internet.…”

Towards Socialized Machines: Emotions and Sense of Humour in Conversational Agents

“…Furthermore, although it was proved that affective states should be analysed as emotion specific (Lerner & Kelter, 2000), most of the behavioural approach methods simply classify them to opposing pairs such as joy-anger, or happiness-sadness (Teixeira et al, 2008). A positive change in this tendency can be seen in text mining and information extraction approaches to emotion estimation (Tokuhisa et al, 2008;Ptaszynski et al, 2009b). However, the lack of standardization often causes inconsistencies in emotion classification.…”

“…However, the lack of standardization often causes inconsistencies in emotion classification. As one of the recent advances in affect analysis, it was shown that Web mining methods can improve the performance of language-based affect analysis systems (Tokuhisa et al, 2008;Ptaszynski et al, 2009b). However, in such methods, although the results of experiments appear to be positive, the two different approaches, the language-syntax based and Web mining based, are mixed.…”