Grounding Emotions in Human-Machine Conversational Systems

Riccardi, Giuseppe; Hakkani‐Tür, Dilek

doi:10.1007/11590323_15

Cited by 15 publications

(9 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We are interested specifically in recognizing negative emotions as some studies, like for example (Riccardi and Hakkani-Tür, 2005), have shown that once the user is in a negative emotional state, it is difficult to guide him out. Furthermore, these bad experiences can also discourage users from employing the system again.…”

Section: Introductionmentioning

confidence: 99%

Influence of contextual information in emotion annotation for spoken dialogue systems

Callejas¹,

López-Cózar²

2008

Speech Communication

View full text Add to dashboard Cite

show abstract

Section: Introductionmentioning

confidence: 99%

Influence of contextual information in emotion annotation for spoken dialogue systems

Callejas¹,

López-Cózar²

2008

Speech Communication

View full text Add to dashboard Cite

show abstract

“…Furthermore, information on the caller's emotional state may be used to predict system error rates. Riccardi and Hakkani-Tür [20] investigate how the user's emotional state affects the accuracy of the AT&T "How May I Help You?" spoken dialogue system and conclude that the detection of the caller's emotional state may be beneficial for the adaptation of the system's dialogue strategies.…”

Section: Applicationsmentioning

confidence: 99%

Automatic Recognition of Emotions from Speech: A Review of the Literature and Recommendations for Practical Realisation

Vogt

André

Wagner

Affect and Emotion in Human-Computer Interaction

103

View full text Add to dashboard Cite

Abstract. In this article we give guidelines on how to address the major technical challenges of automatic emotion recognition from speech in human-computer interfaces, which include audio segmentation to find appropriate units for emotions, extraction of emotion relevant features, classification of emotions, and training databases with emotional speech. Research so far has mostly dealt with offline evaluation of vocal emotions, and online processing has hardly been addressed. Online processing is, however, a necessary prerequisite for the realization of human-computer interfaces that analyze and respond to the user's emotions while he or she is interacting with an application. By means of a sample application, we demonstrate how the challenges arising from online processing may be solved. The overall objective of the paper is to help readers to assess the feasibility of human-computer interfaces that are sensitive to the user's emotional voice and to provide them with guidelines of how to technically realize such interfaces.

show abstract

“…This idea is questionable because users would not have formulated their opinions of the system at least during the first few turns. Moreover, if a user felt negatively towards a system at the beginning of the interaction, the user would tend to remain in that negative state throughout the whole interaction, as pointed out in Riccardi and Hakkani-Tü r (2005). This indicates that if users give ratings on a turn basis, low ratings might be acquired for all the turns that follow a particular problematic turn without them waiting till the end of the interaction in order to formulate a sensible opinion of the system.…”

Section: Related Workmentioning

confidence: 99%

A satisfaction-based model for affect recognition from conversational features in spoken dialog systems

Lutfi

Fernández-Martínez

Lucas-Cuesta

et al. 2013

Speech Communication

View full text Add to dashboard Cite

Detecting user affect automatically during real-time conversation is the main challenge towards our greater aim of infusing social intelligence into a natural-language mixed-initiative High-Fidelity (Hi-Fi) audio control spoken dialog agent. In recent years, studies on affect detection from voice have moved on to using realistic, non-acted data, which is subtler. However, it is more challenging to perceive subtler emotions and this is demonstrated in tasks such as labeling and machine prediction. This paper attempts to address part of this challenge by considering the role of user satisfaction ratings and also conversational/dialog features in discriminating contentment and frustration, two types of emotions that are known to be prevalent within spoken human-computer interaction. However, given the laboratory constraints, users might be positively biased when rating the system, indirectly making the reliability of the satisfaction data questionable. Machine learning experiments were conducted on two datasets, users and annotators, which were then compared in order to assess the reliability of these datasets. Our results indicated that standard classifiers were significantly more successful in discriminating the abovementioned emotions and their intensities (reflected by user satisfaction ratings) from annotator data than from user data. These results corroborated that: first, satisfaction data could be used directly as an alternative target variable to model affect, and that they could be predicted exclusively by dialog features. Second, these were only true when trying to predict the abovementioned emotions using annotator's data, suggesting that user bias does exist in a laboratory-led evaluation.

show abstract

Grounding Emotions in Human-Machine Conversational Systems

Cited by 15 publications

References 9 publications

Influence of contextual information in emotion annotation for spoken dialogue systems

Influence of contextual information in emotion annotation for spoken dialogue systems

Automatic Recognition of Emotions from Speech: A Review of the Literature and Recommendations for Practical Realisation

A satisfaction-based model for affect recognition from conversational features in spoken dialog systems

Contact Info

Product

Resources

About