Classification of Complex Emotions Using EEG and Virtual Environment: Proof of Concept and Therapeutic Implication

Abstract

During the last decades, neurofeedback training for emotional self-regulation has received significant attention from scientific and clinical communities. Most studies have investigated emotions using functional magnetic resonance imaging (fMRI), including the real-time application in neurofeedback training. However, the electroencephalogram (EEG) is a more suitable tool for therapeutic application. Our study aims at establishing a method to classify discrete complex emotions (e.g., tenderness and anguish) elicited through a near-immersive scenario that can be later used for EEG-neurofeedback. EEG-based affective computing studies have mainly focused on emotion classification based on dimensions, commonly using passive elicitation through single-modality stimuli. Here, we integrated both passive and active elicitation methods. We recorded electrophysiological data during emotion-evoking trials, combining emotional self-induction with a multimodal virtual environment. We extracted correlational and time-frequency features, including frontal-alpha asymmetry (FAA), using Complex Morlet Wavelet convolution. Thinking about future real-time applications, we performed within-subject classification using 1-s windows as samples and we applied trial-specific cross-validation. We opted for a traditional machine-learning classifier with low computational complexity and sufficient validation in online settings, the Support Vector Machine. Results of individual-based cross-validation using the whole feature sets showed considerable between-subject variability. The individual accuracies ranged from 59.2 to 92.9% using time-frequency/FAA and 62.4 to 92.4% using correlational features. We found that features of the temporal, occipital, and left-frontal channels were the most discriminative between the two emotions. Our results show that the suggested pipeline is suitable for individual-based classification of discrete emotions, paving the way for future personalized EEG-neurofeedback training.

Music's manifold emotions for mental well-being: a topological approach

“…AI, machine learning and deep learning have started gaining traction in all sorts of fields, not excluding neuroscience; EEG data is a primary mode of AI research into being able to classify/recognize emotions on new data from previous data [119][120][121][122] ; many different forms of machine learning, deep learning and neural networks have been used for this emotion classification purpose [123][124][125][126][127][128][129][130][131] . Leading with AI for music listeningemotional experience research will allow us to form tailored solutions to the mental health crisis we have currently through personalized, convenient therapeutic interventions in hospitals and related clinical settings paired with music listening [132][133][134][135][136][137][138] . Specifically, in future work we can pry into the topologies of subjects' latent neural dynamics with the most naturalistic snapshot of complex music listening and emotional experience: subjects/patients simply listening to the music, with their emotional experience predicted by a highaccuracy deep learning framework 131 , exempt of having to physically report their emotions, which can interfere with the latent neural dynamics of pure music listening and emotional experience, on a physical interface.…”

k-Fold Cross-Validation Can Significantly Over-Estimate True Classification Accuracy in Common EEG-Based Passive BCI Experimental Designs: An Empirical Investigation

“…While this issue has been acknowledged in some BCI papers [5][6][7] and it has been suggested that block-wise cross-validation should be the preferred approach [5,8], the effects of using k-fold CV in such scenarios has not been investigated, and it is not clear how significantly (and under what circumstances) it can overestimate true mental state separability. Furthermore, it is not clear if block-wise CV will actually accurately estimate class separability.…”

Identifying Complex Emotions in Alexithymia Affected Adolescents Using Machine Learning Techniques

“…Eleonora De Filippi et al [ 12 ] used a strategy for distinguishing distinct complex emotions such as tenderness and anguish that can be gleaned from EEG. EEG-based affective computing uses varied proportions of emotion-based classification and is widely utilized in passive elicitation using single-modality stimuli.…”