Recognizing Emotions Induced by Affective Sounds through Heart Rate Variability

Abstract

This paper reports on how emotional states elicited by affective sounds can be effectively recognized by means of estimates of Autonomic Nervous System (ANS) dynamics. Specifically, emotional states are modeled as a combination of arousal and valence dimensions according to the well-known circumplex model of affect, whereas the ANS dynamics is estimated through standard and nonlinear analysis of Heart rate variability (HRV) exclusively, which is derived from the electrocardiogram (ECG). In addition, Lagged Poincaré Plots of the HRV series were also taken into account. The affective sounds were gathered from the International Affective Digitized Sound System and grouped into four different levels of arousal (intensity) and two levels of valence (unpleasant and pleasant). A group of 27 healthy volunteers were administered with these standardized stimuli while ECG signals were continuously recorded. Then, those HRV features showing significant changes (p<0.05 from statistical tests) between the arousal and valence dimensions were used as input of an automatic classification system for the recognition of the four classes of arousal and two classes of valence. Experimental results demonstrated that a quadratic discriminant classifier, tested through Leave-One-Subject-Out procedure, was able to achieve a recognition accuracy of 84.72% on the valence dimension, and 84.26% on the arousal dimension.

Affective Recognition for Multimedia Environments A Review

“…Almost all studies reviewed in this paper did not apply any form of filtering on autonomic nervous system signals 2 . However, to eliminate noise in EEG signals, two key techniques were found to have been employed by those studies reviewed: [32], [33], [34], [35] Sound [29], [30], [31] Real Life Scenarios [36], [37], [38], [39], [40] …”

“…Electroencephalography ( [25], [29], [30], [31], [35], [36], [37], [38], [39] Galvanic Skin Response (GSR) [25], [30], [32], [34], [35], [36], [37], [38], [39] Respiratory ( [33], [40]. This was due to the fact that eye-blinking artefacts are mainly observed in frequencies lower that 4Hz, as people rarely blink more than 4 times a second.…”

“…The studies, which implemented classification algorithms for emotion recognition processes (Table 4), reached an average of 77.91% classification accuracy (±12.76%, with minimum of 50% (random classification) and maximum of 96.5%), calculated through crossvalidation. [25], [30], [34], [35], [36], [37], [38], [39] Heart Rate Spectral Power Ratio 11 [10], [23], [25], [31] 3 Subtraction of the two channels' raw signals 4 GSR low frequency is 0Hz to 2.4Hz 5 Asymmetric Spectral Power Density = ! !…”

“…Sound: Studies, which used sound as their affective stimuli, employed either music [29], [30], or extracts from the International Affective Digital Sound database (IADS [8]) [31]. The average sound duration was reported as 210 seconds (±112.24, minimum 60 and maximum 330 seconds).…”

“…The average stimulation duration was reported as 170 seconds (±75.5, minimum 90 and maximum 240 seconds). [26], [28], [29], [33], [34], [40] [25], [29], [30], [31], [35], [36], [37], [38], [39] [25], [30], [32], [34], [35], [36], [37], [38], [39] [25], [30], [34], [37], [38] [29], [30], [35], [37], [38], [39] …”