Emotion Recognition from Spatio-Temporal Representation of EEG Signals via 3D-CNN with Ensemble Learning Techniques

Yuvaraj, Rajamanickam; Baranwal, Arapan; Prince, A. Amalin; Murugappan, M.; Mohammed, Javeed Shaikh

doi:10.3390/brainsci13040685

Cited by 11 publications

(3 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…3D-CNN ( Yuvaraj et al, 2023 ). By defining the spatial layout of EEG data, the three-dimensional structure of EEG data can be visualized and expressed.…”

Section: Resultsmentioning

confidence: 99%

E-MFNN: an emotion-multimodal fusion neural network framework for emotion recognition

Guo,

Yang,

Lin

et al. 2024

PeerJ Computer Science

View full text Add to dashboard Cite

Emotional recognition is a pivotal research domain in computer and cognitive science. Recent advancements have led to various emotion recognition methods, leveraging data from diverse sources like speech, facial expressions, electroencephalogram (EEG), electrocardiogram, and eye tracking (ET). This article introduces a novel emotion recognition framework, primarily targeting the analysis of users’ psychological reactions and stimuli. It is important to note that the stimuli eliciting emotional responses are as critical as the responses themselves. Hence, our approach synergizes stimulus data with physical and physiological signals, pioneering a multimodal method for emotional cognition. Our proposed framework unites stimulus source data with physiological signals, aiming to enhance the accuracy and robustness of emotion recognition through data integration. We initiated an emotional cognition experiment to gather EEG and ET data alongside recording emotional responses. Building on this, we developed the Emotion-Multimodal Fusion Neural Network (E-MFNN), optimized for multimodal data fusion to process both stimulus and physiological data. We conducted extensive comparisons between our framework’s outcomes and those from existing models, also assessing various algorithmic approaches within our framework. This comparison underscores our framework’s efficacy in multimodal emotion recognition. The source code is publicly available at https://figshare.com/s/8833d837871c78542b29.

show abstract

“…3D-CNN ( Yuvaraj et al, 2023 ). By defining the spatial layout of EEG data, the three-dimensional structure of EEG data can be visualized and expressed.…”

Section: Resultsmentioning

confidence: 99%

E-MFNN: an emotion-multimodal fusion neural network framework for emotion recognition

Guo,

Yang,

Lin

et al. 2024

PeerJ Computer Science

View full text Add to dashboard Cite

show abstract

“…The statistical features were extracted from four frequency bands and represented in a 2D map band-wise individually; thus, a 3D map was constructed concatenating features from all four frequency bands. Yuvaraj et al 39 also constructed a 3D map staking 2D spatiotemporal representation of EEG signals and then employed a 3D form of CNN for emotion recognition.…”

Section: Related Workmentioning

confidence: 99%

Improved EEG-based emotion recognition through information enhancement in connectivity feature map

Akhand,

Maria,

Kamal

et al. 2023

Sci Rep

View full text Add to dashboard Cite

Electroencephalography (EEG), despite its inherited complexity, is a preferable brain signal for automatic human emotion recognition (ER), which is a challenging machine learning task with emerging applications. In any automatic ER, machine learning (ML) models classify emotions using the extracted features from the EEG signals, and therefore, such feature extraction is a crucial part of ER process. Recently, EEG channel connectivity features have been widely used in ER, where Pearson correlation coefficient (PCC), mutual information (MI), phase-locking value (PLV), and transfer entropy (TE) are well-known methods for connectivity feature map (CFM) construction. CFMs are typically formed in a two-dimensional configuration using the signals from two EEG channels, and such two-dimensional CFMs are usually symmetric and hold redundant information. This study proposes the construction of a more informative CFM that can lead to better ER. Specifically, the proposed innovative technique intelligently combines CFMs’ measures of two different individual methods, and its outcomes are more informative as a fused CFM. Such CFM fusion does not incur additional computational costs in training the ML model. In this study, fused CFMs are constructed by combining every pair of methods from PCC, PLV, MI, and TE; and the resulting fused CFMs PCC + PLV, PCC + MI, PCC + TE, PLV + MI, PLV + TE, and MI + TE are used to classify emotion by convolutional neural network. Rigorous experiments on the DEAP benchmark EEG dataset show that the proposed CFMs deliver better ER performances than CFM with a single connectivity method (e.g., PCC). At a glance, PLV + MI-based ER is shown to be the most promising one as it outperforms the other methods.

show abstract

“…This network demonstrated favorable outcomes on the Sleep EDF-1 dataset. Yuvaraj et al [ 30 ] used a pre-trained 3D-CNN MobileNet model for transfer learning on EEG signals to extract features for emotion recognition.…”

Section: Introductionmentioning

confidence: 99%

EEG Emotion Recognition by Fusion of Multi-Scale Features

Du,

Meng,

Qiu

et al. 2023

Brain Sciences

View full text Add to dashboard Cite

Electroencephalogram (EEG) signals exhibit low amplitude, complex background noise, randomness, and significant inter-individual differences, which pose challenges in extracting sufficient features and can lead to information loss during the mapping process from low-dimensional feature matrices to high-dimensional ones in emotion recognition algorithms. In this paper, we propose a Multi-scale Deformable Convolutional Interacting Attention Network based on Residual Network (MDCNAResnet) for EEG-based emotion recognition. Firstly, we extract differential entropy features from different channels of EEG signals and construct a three-dimensional feature matrix based on the relative positions of electrode channels. Secondly, we utilize deformable convolution (DCN) to extract high-level abstract features by replacing standard convolution with deformable convolution, enhancing the modeling capability of the convolutional neural network for irregular targets. Then, we develop the Bottom-Up Feature Pyramid Network (BU-FPN) to extract multi-scale data features, enabling complementary information from different levels in the neural network, while optimizing the feature extraction process using Efficient Channel Attention (ECANet). Finally, we combine the MDCNAResnet with a Bidirectional Gated Recurrent Unit (BiGRU) to further capture the contextual semantic information of EEG signals. Experimental results on the DEAP dataset demonstrate the effectiveness of our approach, achieving accuracies of 98.63% and 98.89% for Valence and Arousal dimensions, respectively.

show abstract

Emotion Recognition from Spatio-Temporal Representation of EEG Signals via 3D-CNN with Ensemble Learning Techniques

Cited by 11 publications

References 34 publications

E-MFNN: an emotion-multimodal fusion neural network framework for emotion recognition

E-MFNN: an emotion-multimodal fusion neural network framework for emotion recognition

Improved EEG-based emotion recognition through information enhancement in connectivity feature map

EEG Emotion Recognition by Fusion of Multi-Scale Features

Contact Info

Product

Resources

About