Real-Time Recognition of Affective States from Nonverbal Features of Speech and Its Application for Public Speaking Skill Analysis

Abstract

Abstract-This paper presents a new classification algorithm for real-time inference of affect from non-verbal features of speech and applies it to assessing public speaking skills. The classifier identifies simultaneously occurring affective states by recognising correlations between emotions and over 6000 functional-feature combinations. Pairwise classifiers are constructed for 9 classes from the Mind Reading emotion corpus, yielding an average cross-validation accuracy of 89% for the pairwise machines and 86% for the fused machine. The paper also shows a novel application of the classifier for assessing public speaking skills, achieving an average cross-validation accuracy of 81% and a leave-one-speaker-out classification accuracy of 61%. Optimising support vector machine coefficients using grid parameter search is shown to improve the accuracy by up to 25%. The emotion classifier outperforms previous research on the same emotion corpus and is successfully applied to analyse public speaking skills.

Online feedback system for public speakers

“…To the best of our knowledge, there are two existing systems sharing this purpose [15], [16]. In [16], the authors directly analysed vocal signal of speaker via low-level physical characteristics.…”

“…In [16], the authors directly analysed vocal signal of speaker via low-level physical characteristics. On the other hand, the research in [15] paid more attention on emotion expression of speakers. In fact, this study is mostly emotion detection from voice, rather than a speaker support system.…”

“…Differ from previous systems that take speech signal into account [15], [16], this system focuses on body language of speakers. Other aspects need to be considered when assess a public speaker are the context and content of the presentation.…”

Recent developments in social signal processing

“…The integration of real-time social signal processing into expert systems opens up new venues for this mature field. To give an example, Pfister and Robinson describe a classification scheme for real-time speech assessment, evaluated in the context of public speaking skills [40]. In this application nonverbal speech cues are extracted and used for assigning affective labels (absorbed, excited, interested, joyful, opposed, stressed, sure, thinking, unsure) to short speech segments, as well as for assessing the speech in terms of its perceived qualities (clear, competent, credible, dynamic, persuasive, pleasant), resulting in a novel and useful coaching scenario.…”

Intelligent Presentation Skills Trainer Analyses Body Movement

“…In the literature, there are several approaches toward the automatic recognition of nonverbal cues from presenters, such as [7][8][9][10]. These approaches analyze some vocal and visual channels of presenters, thus can provide them with the information about their performance.…”