Flexible, Robust and Dynamic Dialogue Modeling with a Speech Dialogue Interface for Controlling a Hi-Fi Audio System

Lecture Notes in Computer Science

Casanova-García

et al. 2012

Abstract. We describe the work on infusion of emotion into limitedtask autonomous spoken conversational agents (SCAs) situated in the domestic environment, using a Need-inspired task-independent Emotion model (NEMO). In order to demonstrate the generation of affect through the use of the model, we describe the work of integrating it with a naturallanguage mixed-initiative HiFi-control SCA. NEMO and the host system communicates externally, removing the need for the Dialog Manager to be modified as done in most existing dialog systems, in order to be adaptive. We also summarize the work on automatic affect prediction, namely frustration and contentment from dialog features, a non-conventional source, in the attempt of moving towards a more user-centric approach.

Section: Infusing Emotions Into the Hifi Agentmentioning

confidence: 99%

“…The original non-adaptive HiFi SCA version used a non-emotional commercial TTS. A detailed architecture of the non-adaptive system is given in [8]. In converting the HiFi SCA into HiFi Spoken Conversational Agent an affect-adaptive system, its existing components were not modified.…”

Section: Architecture Of Hifi-nemomentioning

confidence: 99%

I Feel You: Towards Affect-Sensitive Domotic Spoken Conversational Agents

Lecture Notes in Computer Science

Casanova-García

et al. 2012

“…This paper presents a summary of the the NEMOHIFI 1 * Corresponding Author 1 The work leading to these results has received funding from the European Union under grant agreement nr. 287678.…”

Section: Introductionmentioning

confidence: 99%

Nemohifi

Proceedings of the 15th ACM on International Conference on Multimodal Interaction

Lorenzo-Trueba

et al. 2013

Self Cite

This demo concerns a recently developed prototype of an emotionally-sensitive autonomous HiFi Spoken Conversational Agent, called NEMOHIFI. The baseline agent was developed by the Speech Technology Group (GTH) and has recently been integrated with an emotional engine called NEMO (Need-inspired Emotional Model) to enable it to adapt to users' emotion and respond to the users using appropriate expressive speech. NEMOHIFI controls and manages the HiFi audio system, and for end users, its functions equate a remote control, except that instead of clicking, the user interacts with the agent using voice. A pairwise comparison between the baseline (non-adaptive) and NEMO-HIFI is also presented.

“…At first glance this may seem counter-intuitive mainly because contextual information was intended to expedite task delivery by introducing a repair strategy to reduce the number of system requests (Fernández-Martínez et al, 2010b;Fernández-Martínez et al, 2010a). Specifically, any ambiguous situation would be handled in such way that the system would recover the missing information from the dialog context in an attempt to deliver the task.…”

Section: Significant Predictors Of Gssmentioning

confidence: 99%

“…The HiFi agent is a proprietary system of GTH (details in Fernández-Martínez et al (2010a) agent controls and manages the HiFi audio system, and for end users, its functions equate a remote control (select a CD, track or radio channel, record music, change channels etc. ), except that instead of clicking, the user interacts with the agent using voice.…”

Section: Introductionmentioning

confidence: 99%

A satisfaction-based model for affect recognition from conversational features in spoken dialog systems

Lucas-Cuesta

et al. 2013

Speech Communication

Self Cite

Detecting user affect automatically during real-time conversation is the main challenge towards our greater aim of infusing social intelligence into a natural-language mixed-initiative High-Fidelity (Hi-Fi) audio control spoken dialog agent. In recent years, studies on affect detection from voice have moved on to using realistic, non-acted data, which is subtler. However, it is more challenging to perceive subtler emotions and this is demonstrated in tasks such as labeling and machine prediction. This paper attempts to address part of this challenge by considering the role of user satisfaction ratings and also conversational/dialog features in discriminating contentment and frustration, two types of emotions that are known to be prevalent within spoken human-computer interaction. However, given the laboratory constraints, users might be positively biased when rating the system, indirectly making the reliability of the satisfaction data questionable. Machine learning experiments were conducted on two datasets, users and annotators, which were then compared in order to assess the reliability of these datasets. Our results indicated that standard classifiers were significantly more successful in discriminating the abovementioned emotions and their intensities (reflected by user satisfaction ratings) from annotator data than from user data. These results corroborated that: first, satisfaction data could be used directly as an alternative target variable to model affect, and that they could be predicted exclusively by dialog features. Second, these were only true when trying to predict the abovementioned emotions using annotator's data, suggesting that user bias does exist in a laboratory-led evaluation.