Transformer-based ensemble deep learning model for EEG-based emotion recognition

Si, Xiaopeng; Huang, Dong; Sun, Yulin; Huang, S. W.; He, Hangen; Ming, Dong

doi:10.26599/bsa.2023.9050016

Cited by 3 publications

(2 citation statements)

References 57 publications

(56 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The transformer-based model emerges as a power-ful approach, capable of learning discriminative spatial information extending from the electrode level to the brain-region level, in the pursuit of improving the ability to capture EEG spatial dependencies and enhance the accuracy of emotion recognition. [22,23].…”

Section: Exploring Past Work: a Brief Literature Reviewmentioning

confidence: 99%

Integrated Modeling for In-Depth EEG Based Emotional State Analysis via Convolutional-Transformer Fusion

Ali,

Kang,

Himral

2023

Preprint

View full text Add to dashboard Cite

This study introduces a hybrid model designed for the predic- tion of emotional states derived from electroencephalogram (EEG) data, employing an amalgamation of convolutional and trans- former layers. The architectural framework of the model is metic- ulously structured to facilitate concurrent assimilation of local pat- terns and long-range dependencies inherent in EEG data, thereby augmenting its discernment of nuanced emotional experiences. The investigation presented herein undertakes a comprehensive explo- ration of the fusion technique, with a primary focus on dis- cerning three elemental emotional dimensions: Arousal, Valence, and Dominance, in addition to their concurrent combinations. The research methodology encompasses an in-depth evaluation of the model’s performance across these diverse emotional states, encompassing the intricate task of simultaneous Valence-Arousal (VA) prediction. Furthermore, the study extends its purview to encompass the intricate Valence-Arousal-Dominance (VAD) space, thereby providing a thorough analysis of the model’s efficacy. To articulate the model’s discriminative efficacy, this study meticu- lously presents the detailed F1 scores corresponding to each emotional state classification: Arousal (96.8), Valence (97.3), Valence-Arousal (VA) simultaneously (95.6), and Valence-Arousal-Dominance simultaneously (94.9). These scores serve as a testament to the model’s robust per- formance across diverse emotional dimensions. Importantly, to fortify the credibility of our findings, rigorous experimentation has been con- ducted on the DEAP dataset, unveiling noteworthy results even in scenarios involving simultaneous recognition of multiple emotional states.

show abstract

Section: Exploring Past Work: a Brief Literature Reviewmentioning

confidence: 99%

Integrated Modeling for In-Depth EEG Based Emotional State Analysis via Convolutional-Transformer Fusion

Ali,

Kang,

Himral

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Si et al [6] proposed a Transformer-based ensemble (TBEM) model that comprised a pure convolutional neural network (CNN) and a cascaded CNN-Transformer hybrid model for EEG-based emotion recognition. By leveraging ensemble learning and re-referencing preprocessing, the proposed TBEM method was able to effectively discriminate between eight emotional states and outperform other competing methods in the affective BCI track.…”

Section: Introductionmentioning

confidence: 99%

BCI Controlled Robot Contest on the 50th Anniversary of Brain-Computer Interfaces

Liu,

Chen,

Chen

et al. 2023

Brain Science Advances

View full text Add to dashboard Cite

A novel methodology for emotion recognition through 62-lead EEG signals: multilevel heterogeneous recurrence analysis

Wang,

Chen,

Imamura

et al. 2024

Front. Physiol.

View full text Add to dashboard Cite

ObjectiveRecognizing emotions from electroencephalography (EEG) signals is a challenging task due to the complex, nonlinear, and nonstationary characteristics of brain activity. Traditional methods often fail to capture these subtle dynamics, while deep learning approaches lack explainability. In this research, we introduce a novel three-phase methodology integrating manifold embedding, multilevel heterogeneous recurrence analysis (MHRA), and ensemble learning to address these limitations in EEG-based emotion recognition.ApproachThe proposed methodology was evaluated using the SJTU-SEED IV database. We first applied uniform manifold approximation and projection (UMAP) for manifold embedding of the 62-lead EEG signals into a lower-dimensional space. We then developed MHRA to characterize the complex recurrence dynamics of brain activity across multiple transition levels. Finally, we employed tree-based ensemble learning methods to classify four emotions (neutral, sad, fear, happy) based on the extracted MHRA features.Main resultsOur approach achieved high performance, with an accuracy of 0.7885 and an AUC of 0.7552, outperforming existing methods on the same dataset. Additionally, our methodology provided the most consistent recognition performance across different emotions. Sensitivity analysis revealed specific MHRA metrics that were strongly associated with each emotion, offering valuable insights into the underlying neural dynamics.SignificanceThis study presents a novel framework for EEG-based emotion recognition that effectively captures the complex nonlinear and nonstationary dynamics of brain activity while maintaining explainability. The proposed methodology offers significant potential for advancing our understanding of emotional processing and developing more reliable emotion recognition systems with broad applications in healthcare and beyond.

show abstract

Transformer-based ensemble deep learning model for EEG-based emotion recognition

Cited by 3 publications

References 57 publications

Integrated Modeling for In-Depth EEG Based Emotional State Analysis via Convolutional-Transformer Fusion

Integrated Modeling for In-Depth EEG Based Emotional State Analysis via Convolutional-Transformer Fusion

BCI Controlled Robot Contest on the 50th Anniversary of Brain-Computer Interfaces

A novel methodology for emotion recognition through 62-lead EEG signals: multilevel heterogeneous recurrence analysis

Contact Info

Product

Resources

About