EEG-Based Emotion Recognition Using Spatial-Temporal Graph Convolutional LSTM With Attention Mechanism

Feng, Lin; Cheng, Cheng; Zhao, Mi; Deng, Huiyuan; Zhang, Yong

doi:10.1109/jbhi.2022.3198688

Cited by 68 publications

(19 citation statements)

References 64 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zhang et al (2019a) adopted a multi-directional recurrent neural network to construct a complete spatial graph for modeling dependencies among the electrodes. Feng et al (2022) proposed a spatial-graph convolutional network to fully leverage the EEG spatial information. In addition, a brain connectivity analysis also accurately captures the correlations among the channels.…”

Section: Related Work 21 Spatial Learning Of Eeg Signals For Emotion ...mentioning

confidence: 99%

Cerebral asymmetry representation learning-based deep subdomain adaptation network for electroencephalogram-based emotion recognition

Wang,

Wan

et al. 2024

Physiol. Meas.

View full text Add to dashboard Cite

Objective. Extracting discriminative spatial information from multiple electrodes is a crucial and challenging problem for Electroencephalogram (EEG)-based emotion recognition. Additionally, the domain shift caused by the individual differences would degrade the performance of cross-subject EEG classification. Approach. To deal with the problems, we propose the Cerebral Asymmetry Representation Learning based Deep Subdomain Adaptation Network (CARL-DSAN) to enhance the cross-subject EEG-based emotion recognition. Specifically, the CARL module is inspired by the neuroscience findings that asymmetrical activations of left and right brain hemispheres occur during cognitive and affective processes. In the CARL module, we introduce a novel two-step strategy for extracting discriminative features through intra-hemisphere spatial learning and asymmetry representation learning. Moreover, the transformer encoders within the CARL module could emphasize the contributive electrodes and electrode pairs. Subsequently, the DSAN module, known for its superior performance over global domain adaption, is adopted to mitigate domain shift and further improve the cross-subject performance by aligning relevant subdomains which share the same class samples. Main Results. To validate the effectiveness of the CARL-DSAN, we conduct the subject-independent experiments on the DEAP database, achieving the accuracies of 68.67% and 67.11% for arousal and valence classification respectively, and corresponding accuracies of 67.70% and 67.18% on the MAHNOB-HCI database. Significance. The results demonstrate that CARL-DSAN could achieve outstanding cross-subject performance in both arousal and valence classification.

show abstract

Section: Related Work 21 Spatial Learning Of Eeg Signals For Emotion ...mentioning

confidence: 99%

Cerebral asymmetry representation learning-based deep subdomain adaptation network for electroencephalogram-based emotion recognition

Wang,

Wan

et al. 2024

Physiol. Meas.

View full text Add to dashboard Cite

show abstract

“…A graph convolutional broad network was designed to explore the deeper-level information of graph-structured data and achieved high performance in EEG-based emotion recognition (Zhang et al, 2019 ). Li et al proposed a Multi-Domain Adaptive Graph Convolutional Network (MD-AGCN), fusing the knowledge of both the frequency domain and the temporal domain to fully utilize the complementary information of EEG signals (Li R. et al, 2021 ) designed a model called ST-GCLSTM, which utilizes spatial attention to modify adjacency matrices to adaptively learn the intrinsic connection among different EEG channels (Feng et al, 2022 ).…”

Section: Related Workmentioning

confidence: 99%

STGATE: Spatial-temporal graph attention network with a transformer encoder for EEG-based emotion recognition

Pan

Huang

et al. 2023

Front. Hum. Neurosci.

View full text Add to dashboard Cite

Electroencephalogram (EEG) is a crucial and widely utilized technique in neuroscience research. In this paper, we introduce a novel graph neural network called the spatial-temporal graph attention network with a transformer encoder (STGATE) to learn graph representations of emotion EEG signals and improve emotion recognition performance. In STGATE, a transformer-encoder is applied for capturing time-frequency features which are fed into a spatial-temporal graph attention for emotion classification. Using a dynamic adjacency matrix, the proposed STGATE adaptively learns intrinsic connections between different EEG channels. To evaluate the cross-subject emotion recognition performance, leave-one-subject-out experiments are carried out on three public emotion recognition datasets, i.e., SEED, SEED-IV, and DREAMER. The proposed STGATE model achieved a state-of-the-art EEG-based emotion recognition performance accuracy of 90.37% in SEED, 76.43% in SEED-IV, and 76.35% in DREAMER dataset, respectively. The experiments demonstrated the effectiveness of the proposed STGATE model for cross-subject EEG emotion recognition and its potential for graph-based neuroscience research.

show abstract

“…3DCANN is composed of spatial-temporal feature extraction and EEG channel attention weight learning, which can better extract the dynamic relationships and internal spatial relationships between multi-channel EEG signals in continuous time periods; but the classification model used is too simple in the pattern recognition stage (Liu et al 2021). Feng et al designed a model consisting of spatial graph convolutional networks and attention-enhanced bi-directional LSTM to extract representative spatial-temporal features from multiple EEG channels; but their model ignored representative EEG features such as DE features, PSD features, etc (Feng et al 2022). Li et al proposed a multi-domain adaptive graph convolutional network, which integrates frequency-domain and time-domain features to make full use of complementary information of EEG signals; yet it ignores spatial features and does not achieve favorable recognition performance .…”

Section: Deep Neural Network For Emotion Recognitionmentioning

confidence: 99%

Spatial–temporal features-based EEG emotion recognition using graph convolution network and long short-term memory

Zheng

et al. 2023

Physiol. Meas.

View full text Add to dashboard Cite

Objective: emotion recognition on the basis of electroencephalography (EEG) signals has received a significant amount of attention in the areas of cognitive science and human-computer interaction (HCI). However, most existing studies either focus on one-dimensional EEG data ignoring the relationship between channels, or only extract time-frequency features while not involving spatial features. Approach: we develop spatial-temporal features based EEG emotion recognition using graph convolution network (GCN) and long short-term memory (LSTM) named ERGL. Firstly, the one dimensional EEG vector is converted into a two-dimensional mesh matrix, so that the matrix configuration corresponds to the distribution of brain regions at EEG electrode locations, thus to represent the spatial correlation between multiple adjacent channels in a better way. Secondly, the GCN and LSTM are employed together to extract spatial-temporal features, and GCN is used to extract spatial features, while LSTM units are applied to extract temporal features. Finally, the softmax layer is applied to emotion classification. Main results: extensive experiments are conducted on the DEAP and SEED datasets. The classification results of accuracy, precision and F-score for valence and arousal dimensions on DEAP achieved 90.67% and 90.33%, 92.38% and 91.72% and 91.34% and 90.86%, respectively. The accuracy, precision and F-score of positive, neutral and negative classification reached 94.92%, 95.34% and 94.17%, respectively on SEED dataset. Significance: the above results demonstrate the ERGL method is encouraging in comparison to the state-of-the-art recognition researches.

show abstract

EEG-Based Emotion Recognition Using Spatial-Temporal Graph Convolutional LSTM With Attention Mechanism

Cited by 68 publications

References 64 publications

Cerebral asymmetry representation learning-based deep subdomain adaptation network for electroencephalogram-based emotion recognition

Cerebral asymmetry representation learning-based deep subdomain adaptation network for electroencephalogram-based emotion recognition

STGATE: Spatial-temporal graph attention network with a transformer encoder for EEG-based emotion recognition

Spatial–temporal features-based EEG emotion recognition using graph convolution network and long short-term memory

Contact Info

Product

Resources

About