Electroencephalography Amplitude Modulation Analysis for Automated Affective Tagging of Music Video Clips

Abstract

The quantity of music content is rapidly increasing and automated affective tagging of music video clips can enable the development of intelligent retrieval, music recommendation, automatic playlist generators, and music browsing interfaces tuned to the users' current desires, preferences, or affective states. To achieve this goal, the field of affective computing has emerged, in particular the development of so-called affective brain-computer interfaces, which measure the user's affective state directly from measured brain waves using non-invasive tools, such as electroencephalography (EEG). Typically, conventional features extracted from the EEG signal have been used, such as frequency subband powers and/or inter-hemispheric power asymmetry indices. More recently, the coupling between EEG and peripheral physiological signals, such as the galvanic skin response (GSR), have also been proposed. Here, we show the importance of EEG amplitude modulations and propose several new features that measure the amplitude-amplitude cross-frequency coupling per EEG electrode, as well as linear and non-linear connections between multiple electrode pairs. When tested on a publicly available dataset of music video clips tagged with subjective affective ratings, support vector classifiers trained on the proposed features were shown to outperform those trained on conventional benchmark EEG features by as much as 6, 20, 8, and 7% for arousal, valence, dominance and liking, respectively. Moreover, fusion of the proposed features with EEG-GSR coupling features showed to be particularly useful for arousal (feature-level fusion) and liking (decision-level fusion) prediction. Together, these findings show the importance of the proposed features to characterize human affective states during music clip watching.

Electroencephalogram Profiles for Emotion Identification over the Brain Regions Using Spectral, Entropy and Temporal Biomarkers

“…Standard filtering may be an aspect of the preprocessing phase, with EEG signals seeing the introduction of further software filters-band pass and notch filters for example-to carry out this process. As a means of restricting EEG signal frequencies in accordance with [76], a higher cutoff frequency around 64 Hz and a low cutoff of 0.5 Hz (3 dB) was adopted for the band pass filter. A 50 Hz cutoff frequency was adopted for the notch filter; eliminating A/C electricity line interference is the typical reason for doing so [38].…”

Fusion of Motif- and Spectrum-Related Features for Improved EEG-Based Emotion Recognition

“…Moreover, as the DEAP database relies on 9-point scale ratings, it has typically been the case where the midpoint is considered as a threshold, where ratings greater than the threshold are considered “high,” and those below are considered “low”. As was recently emphasized in [4], however, subjects have their own internal biases, thus leading to varying scales for grading and, consequently, different thresholds per participant. For example, as reported in [4], by using a midpoint threshold value of 5, a 60/40 ratio of high/low levels was obtained across all participants.…”

“…For example, affective computing can enable applications in which the machine can learn user preferences based on their reactions to different settings or even become a more effective tutor by assessing the student's emotional/stress states [3]. Automated recommender and tagging systems, in turn, can make use of affect information to better understand user preferences, thus improving system usability [4]. Measuring affective state and engagement levels can also be used by a machine to infer the user's perceived quality of experience [5–9], thus providing the machine with an objective criterion for online optimization.…”

Design of Wearable EEG Devices Specialized for Passive Brain–Computer Interface Applications

“…For instance, the global field synchronization—a measure calculated using EEG signals acquired from the entire brain—was reported to be effective for estimating users’ emotional valence [ 24 ]. Additionally, Clerico et al reported that the use of connectivity features between multiple interhemispheric electrode pairs could enhance the overall accuracy of emotion classification [ 25 ]. However, a cumbersome preparation process is generally involved for appropriately wearing the devices, as the electrodes must remain in tight contact with the scalp surface in the hair-bearing area [ 26 ].…”