On the use of prosody in automatic dialogue understanding

Nöth, Elmar; Batliner, Anton; Warnke, Volker; Haas, Jürgen; Boros, Manuela; Buckow, Jan; Huber, Richard; Gallwitz, Florian; Nutt, M.; Niemann, Heinrich

doi:10.1016/s0167-6393(01)00025-5

Cited by 23 publications

(8 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Pitch, intensity, duration, voice quality, and long term spectra are gathered under the umbrella of supra-segmental or prosodic features [77,90,99,96,140,166,92,241].…”

Section: Acoustic Low-level Descriptorsmentioning

confidence: 99%

Recognising realistic emotions and affect in speech: State of the art and lessons learnt from the first challenge

Schuller

Batliner

Steidl

et al. 2011

Speech Communication

615

329

View full text Add to dashboard Cite

More than a decade has passed since research on automatic recognition of emotion from speech has become a new field of research in line with its 'big brothers' speech and speaker recognition. This article attempts to provide a short overview on where we are today, how we got there and what this can reveal us on where to go next and how we could arrive there. In a first part, we address the basic phenomenon reflecting the last fifteen years, commenting on databases, modelling and annotation, the unit of analysis and prototypicality. We then shift to automatic processing including discussions on features, classification, robustness, evaluation, and implementation and system integration. From there we go to the first comparative challenge on emotion recognition from speech -the INTERSPEECH 2009 Emotion Challenge, organised by (part of) the authors, including the description of the Challenge's database, Sub-Challenges, participants and their approaches, the winners, and the fusion of results to the actual learnt lessons before we finally address the ever-lasting problems and future promising attempts.Keywords: emotion, affect, automatic classification, feature types, feature selection, noise robustness, adaptation, standardisation, usability, evaluation Setting the SceneThis special issue will address new approaches towards dealing with the processing of realistic emotions in speech, and this overview article will give an account of the state-of-the-art, of the lacunas in this field, and of promising approaches towards overcoming shortcomings in modelling and recognising realistic emotions. We will also report on the first emotion challenge at INTERSPEECH 2009, constituting the initial impetus of this special issue; to end with, we want to sketch future strategies and applications, trying to answer the question 'Where to go from here?'The article is structured as follows: we first deal with the basic phenomenon briefly reflecting the last fifteen years, commenting on databases, modelling and annotation, the unit of analysis and prototypicality. We then proceed to automatic processing (sec. 2) including discussions on features, classification, robustness, evaluation, and implementation and system integration. From there we go to the the first Emotion Challenge (sec. 3) including the description of the Challenge's database, Sub-Challenges, participants and their approaches, the winners, and the fusion of results to the lessons learnt, before concluding this article (sec. 4).

show abstract

“…Pitch, intensity, duration, voice quality, and long term spectra are gathered under the umbrella of supra-segmental or prosodic features [77,90,99,96,140,166,92,241].…”

Section: Acoustic Low-level Descriptorsmentioning

confidence: 99%

Recognising realistic emotions and affect in speech: State of the art and lessons learnt from the first challenge

Schuller

Batliner

Steidl

et al. 2011

Speech Communication

615

329

View full text Add to dashboard Cite

show abstract

“…Syntactic and prosodic features are often used as indicators of utterance endings (e.g. Shriberg et al, 1998;Stolcke et al, 2000;Nöth et al, 2002), but are in general not very reliable. In the case of nonverbal or multimodal communication, the notion of an utterance as a linguistically defined unit is even less clear.…”

Section: Introductionmentioning

confidence: 99%

Multifunctionality in dialogue

Bunt

2011

Computer Speech & Language

View full text Add to dashboard Cite

This paper studies the multifunctionality of dialogue utterances, i.e. the phenomenon that utterances in dialogue often have more than one communicative function. It is argued that this phenomenon can be explained by analyzing the participation in dialogue as involving the performance of several types of activity in parallel, relating to different dimensions of communication. The multifunctionality of dialogue utterances is studied by (1) redefining the notion of 'utterance' in a rigorous manner (calling the revised notion 'functional segment'), and (2) empirically investigating the multifunctionality of functional segments in a corpus of dialogues, annotated with a rich, multidimensional annotation schema. It is shown that, when communicative functions are assigned to functional segments, thereby eliminating every form of segmentation-related multifunctionality, an average multifunctionality is found between 1.8 and 3.6, depending on what is considered to count as a segment's communicative function. Moreover, a good understanding of the nature of the relations among the various multiple functions that a segment may have, and how functional segments relate to other units in dialogue segmentation, opens the way for defining a multidimensional computational update semantics for dialogue interpretation.

show abstract

“…Gallwitz et al [24] also found in sentences' prosodic limits a new manner to segment and identify words. Lastly, prosody helps in dialog acts determination without independent syntactic and semantic analysis [25] and in games control contextualization [26] [27].…”

Section: Related Workmentioning

confidence: 99%

A Connectionist Approach to Speech Understanding

Müller

Siqueira

Navaux

2006

The 2006 IEEE International Joint Conference on Neural Network Proceedings

View full text Add to dashboard Cite

This paper argues that connectionist systems are a good approach to implement a speech understanding computational model. In this direction, we propose SUM, a Speech Understanding Model, which is a software architecture based on recent neurocognitive researches. The SUM´s computational implementation applies wavelets transforms to speech signal processing and connectionist models to syntactic parsing and prosodic-semantic mapping. This approach enables a system of analysis by compensation, when the syntactic analysis does not offer a good reliable level, it is possible to evaluate prosodicsemantic analysis, such as in human speech understanding. I. INTRODUCTIONThe studies of computational systems, which research oral human-computer communication, are commonly called Spoken Language Understanding (SLU) Systems. This kind of system is derived from two joint technologies: Automatic Speech Recognition (ASR) and Natural Language Processing (NLP). These two technologies are complementary. The natural language can help the speech recognition through information on syntax and semantics. Furthermore, the speech recognition can improve the understanding of the language with contextual information, such as the intonation of the words (prosody) [1].Current SLU systems lack technology based on biological studies and there is large interference of the designer in the system configuration. Therefore, it is necessary a study of the biological processes involved in the speech understanding, which will lead to a coherent and valid computational representation of such processes. Our work supports the possibility of computational systems integration to represent the speech understanding process, which will improve the SLU systems development.In fact, recent neurocognitive researches present new approaches to base computational representations. The neurocognitive model developed by Friederici [2] demonstrates a parallel and complementary process that is performed by syntactic and semantics analyses. In this direction, we have proposed a new computational architecture to allow a more robust approach to spoken language processing [3] [4].This work argues that it is possible to unify several computational systems to represent the speech understanding process. Thus, we propose SUM (figure 1), a Speech Understanding Model based on a neurocognitive model of auditory sentence (section III). Through SUM, we search a computational model that works by compensation: when syntactic analysis fails, semantic analysis can be successful, and vice versa.

show abstract

On the use of prosody in automatic dialogue understanding

Cited by 23 publications

References 30 publications

Recognising realistic emotions and affect in speech: State of the art and lessons learnt from the first challenge

Recognising realistic emotions and affect in speech: State of the art and lessons learnt from the first challenge

Multifunctionality in dialogue

A Connectionist Approach to Speech Understanding

Contact Info

Product

Resources

About