Bimodal Emotion Recognition using Speech and Physiological Changes

Kim, Jong-Hwa

doi:10.5772/4754

Cited by 102 publications

(80 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Speech and biosignals are par excellence suitable for personalized and ubiquitous emotion-aware computing technology. However, surprisingly, this combination has hardly been explored; except for the author's own work [97], the only work the author is acquainted with that applied this combination is that of Kim et al [41,42,44,45]. Processing both signals in parallel can, however, be done conveniently, as is illustrated by this study; see also Fig.…”

Section: The Five Issues Under Investigationmentioning

confidence: 93%

See 1 more Smart Citation

Ubiquitous emotion-aware computing

Broek

2011

Pers Ubiquit Comput

View full text Add to dashboard Cite

Emotions are a crucial element for personal and ubiquitous computing. What to sense and how to sense it, however, remain a challenge. This study explores the rare combination of speech, electrocardiogram, and a revised Self-Assessment Mannequin to assess people's emotions. 40 people watched 30 International Affective Picture System pictures in either an office or a living-room environment. Additionally, their personality traits neuroticism and extroversion and demographic information (i.e., gender, nationality, and level of education) were recorded. The resulting data were analyzed using both basic emotion categories and the valence-arousal model, which enabled a comparison between both representations. The combination of heart rate variability and three speech measures (i.e., variability of the fundamental frequency of pitch (F0), intensity, and energy) explained 90% (p \ .001) of the participants' experienced valence-arousal, with 88% for valence and 99% for arousal (ps \ .001). The six basic emotions could also be discriminated (p \ .001), although the explained variance was much lower: 18-20%. Environment (or context), the personality trait neuroticism, and gender proved to be useful when a nuanced assessment of people's emotions was needed. Taken together, this study provides a significant leap toward robust, generic, and ubiquitous emotion-aware computing.

show abstract

Section: The Five Issues Under Investigationmentioning

confidence: 93%

“…To the author's knowledge, only two groups have reported on this combination: Kim et al [41,42,44,45] and the current author and colleagues [97]. A possible explanation is the lack of knowledge of the application of this combination of measures.…”

Section: Ubiquitous Signals Of Emotionmentioning

confidence: 99%

Ubiquitous emotion-aware computing

Broek

2011

Pers Ubiquit Comput

View full text Add to dashboard Cite

show abstract

“…[3]), and foreign speech (e.g. [4]). Inevitably, using these methods to mask linguistic content will affect certain acoustic properties that may characterise the emotion present e.g.…”

Section: Introductionmentioning

confidence: 99%

Judging Emotion from Low-Pass Filtered Naturalistic Emotional Speech

Snel¹,

Cullen²

2013

2013 Humaine Association Conference on Affective Computing and Intelligent Interaction

View full text Add to dashboard Cite

Abstract-In speech, low frequency regions play a significant role in paralinguistic communication such as the conveyance of emotion or mood. The extent to which lower frequencies signify or contribute to affective speech is still an area for investigation. To investigate paralinguistic cues, and remove interference from linguistic cues, researchers can low-pass filter the speech signal on the assumption that certain acoustic cues characterizing affect are still discernible. Low-pass filtering is a practical technique to investigate paralinguistic phenomena, and is used here to investigate the inference of naturalistic emotional speech. This paper investigates how listeners perceive the level of Activation, and evaluate negative and positive levels, on low-pass filtered naturalistic speech, which has been developed through the use of Mood Inducing Procedures.

show abstract

“…The result for their decision level fusion was slightly lower than the feature level. Jonghwa Kim evaluated feature level, decision level, and hybrid fusion performance integrating multichannel physiological signals and speech signal for detecting valance and arousal using linear discriminant analysis (LDA) classifier [16]. Their fusion scheme reported results for feature, decision, and hybrid fusion, where the performance for the feature fusion was highest.…”

Section: Introductionmentioning

confidence: 99%

Classification of affects using head movement, skin color features and physiological signals

Monkaresi

Hussain

Calvo

2012

2012 IEEE International Conference on Systems, Man, and Cybernetics (SMC)

View full text Add to dashboard Cite

Abstract-The automated detection of emotions opens the possibility to new applications in areas such as education, mental health and entertainment. There is an increasing interest on detection techniques that combine multiple modalities. In this study, we introduce automated techniques to detect users' affective states from a fusion model of facial videos and physiological measures. The natural behavior expressed on faces and their physiological responses were recorded from subjects (N=20) while they viewed images from the International Affective Picture System (IAPS). This paper provides a direct comparison between user-dependent, gender-specific, and combined-subject models for affect classification. The analysis indicates that the accuracy of the fusion model (head movement, facial color, and physiology) was statistically higher than the best individual modality for spontaneous affect expressions.

show abstract

Bimodal Emotion Recognition using Speech and Physiological Changes

Cited by 102 publications

References 21 publications

Ubiquitous emotion-aware computing

Ubiquitous emotion-aware computing

Judging Emotion from Low-Pass Filtered Naturalistic Emotional Speech

Classification of affects using head movement, skin color features and physiological signals

Contact Info

Product

Resources

About