STILN: A novel spatial-temporal information learning network for EEG-based emotion recognition

Tang, Yiheng; Wang, Yongxiong; Zhang, Xiaoli; Wang, Zhe

doi:10.1016/j.bspc.2023.104999

Cited by 15 publications

(5 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Additionally, the number of epochs was set to 100 for the DEAP dataset and 50 for the DREAMER dataset. We conducted subject-dependent experiments, where training data and testing data from the same subjects, with five-fold cross-validation, as in many past works 1 , 7 , 31 , 39 , 41 – 45 .…”

Section: Experiments Results and Analysismentioning

confidence: 99%

“…Two types of subject-dependent experiments, where training data and testing data from the same subjects, with five-fold cross-validation 1 , 7 , 31 , 39 , 41 – 45 were conducted: binary classification and multi-classification. In the first type of subject-dependent experiment, the labels were divided into high and low categories.…”

Section: Experiments Results and Analysismentioning

confidence: 99%

“…Tang, et al 31 proposed the spatial-temporal information learning network to extract discriminative features from EEG signals. The network captures spatial correlations and temporal contexts using a combination of a convolutional neural network, convolutional block attention module, and bidirectional long short-term memory.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

TPRO-NET: an EEG-based emotion recognition method reflecting subtle changes in emotion

Zhang,

Cheng,

Liu

2024

Sci Rep

View full text Add to dashboard Cite

Emotion recognition based on Electroencephalogram (EEG) has been applied in various fields, including human–computer interaction and healthcare. However, for the popular Valence-Arousal-Dominance emotion model, researchers often classify the dimensions into high and low categories, which cannot reflect subtle changes in emotion. Furthermore, there are issues with the design of EEG features and the efficiency of transformer. To address these issues, we have designed TPRO-NET, a neural network that takes differential entropy and enhanced differential entropy features as input and outputs emotion categories through convolutional layers and improved transformer encoders. For our experiments, we categorized the emotions in the DEAP dataset into 8 classes and those in the DREAMER dataset into 5 classes. On the DEAP and the DREAMER datasets, TPRO-NET achieved average accuracy rates of 97.63%/97.47%/97.88% and 98.18%/98.37%/98.40%, respectively, on the Valence/Arousal/Dominance dimension for the subject-dependent experiments. Compared to other advanced methods, TPRO-NET demonstrates superior performance.

show abstract

Section: Experiments Results and Analysismentioning

confidence: 99%

Section: Experiments Results and Analysismentioning

confidence: 99%

See 1 more Smart Citation

TPRO-NET: an EEG-based emotion recognition method reflecting subtle changes in emotion

Zhang,

Cheng,

Liu

2024

Sci Rep

View full text Add to dashboard Cite

show abstract

“…In this work, we adopt subject-independent experiment on DEAP database to evaluate the effectiveness of our model. Following the evaluation metrics commonly utilized in previous research (Li et al 2020, Tang et al 2023, we have adopted accuracy and F1-score as our criteria for assessing classification performance. The leave-one-subject-out cross-validation method is employed to assess the model.…”

Section: Resultsmentioning

confidence: 99%

Joint low-rank tensor fusion and cross-modal attention for multimodal physiological signals based emotion recognition

Wan,

Wang,

Wang

et al. 2024

Physiol. Meas.

View full text Add to dashboard Cite

Objective. Physiological signals based emotion recognition is a prominent research domain in the field of human-computer interaction. Previous studies predominantly focused on unimodal data, giving limited attention to the interplay among multiple modalities. Within the scope of multimodal emotion recognition, integrating the information from diverse modalities and leveraging the complementary information are the two essential issues to obtain the robust representations. Approach. Thus, we propose a intermediate fusion strategy for combining low-rank tensor fusion with the cross-modal attention to enhance the fusion of electroencephalogram (EEG), electrooculogram (EOG), electromyography (EMG), and galvanic skin response (GSR). Firstly, handcrafted features from distinct modalities are individually fed to corresponding feature extractors to obtain latent features. Subsequently, low-rank tensor is fused to integrate the information by the modality interaction representation. Finally, a cross-modal attention module is employed to explore the potential relationships between the distinct latent features and modality interaction representation, and recalibrate the weights of different modalities. And the resultant representation is adopted for emotion recognition. Main Results. Furthermore, to validate the effectiveness of the proposed method, we execute subject-independent experiments within the DEAP dataset. The proposed method has achieved the accuracies of 73.82% and 74.55% for valence and arousal classification. Significance. The results of extensive experiments verify the outstanding performance of the proposed method.

show abstract

“…Results showed that self-supervised learning outperformed fully supervised learning methods in classifying arousal and valence using EEG and electrocardiogram (ECG) data. Tang et al [10] conducted experiments on emotion recognition using EEG data with a proposed model called Spatial-Temporal Information Learning Network (STILN), which achieved an accuracy of 68.31% for arousal and 67.52% for valence. Choi et al [11] proposed an attention-LRCN model that reduced motion artifacts in collected photoplethysmography (PPG) data.…”

Section: Related Workmentioning

confidence: 99%

Deep Learning Models for Stress Analysis in University Students: A Sudoku-Based Study

Chen

Lee

2023

Sensors

View full text Add to dashboard Cite

Due to the phenomenon of “involution” in China, the current generation of college and university students are experiencing escalating levels of stress, both academically and within their families. Extensive research has shown a strong correlation between heightened stress levels and overall well-being decline. Therefore, monitoring students’ stress levels is crucial for improving their well-being in educational institutions and at home. Previous studies have primarily focused on recognizing emotions and detecting stress using physiological signals like ECG and EEG. However, these studies often relied on video clips to induce various emotional states, which may not be suitable for university students who already face additional stress to excel academically. In this study, a series of experiments were conducted to evaluate students’ stress levels by engaging them in playing Sudoku games under different distracting conditions. The collected physiological signals, including PPG, ECG, and EEG, were analyzed using enhanced models such as LRCN and self-supervised CNN to assess stress levels. The outcomes were compared with participants’ self-reported stress levels after the experiments. The findings demonstrate that the enhanced models presented in this study exhibit a high level of proficiency in assessing stress levels. Notably, when subjects were presented with Sudoku-solving tasks accompanied by noisy or discordant audio, the models achieved an impressive accuracy rate of 95.13% and an F1-score of 93.72%. Additionally, when subjects engaged in Sudoku-solving activities with another individual monitoring the process, the models achieved a commendable accuracy rate of 97.76% and an F1-score of 96.67%. Finally, under comforting conditions, the models achieved an exceptional accuracy rate of 98.78% with an F1-score of 95.39%.

show abstract

STILN: A novel spatial-temporal information learning network for EEG-based emotion recognition

Cited by 15 publications

References 39 publications

TPRO-NET: an EEG-based emotion recognition method reflecting subtle changes in emotion

TPRO-NET: an EEG-based emotion recognition method reflecting subtle changes in emotion

Joint low-rank tensor fusion and cross-modal attention for multimodal physiological signals based emotion recognition

Deep Learning Models for Stress Analysis in University Students: A Sudoku-Based Study

Contact Info

Product

Resources

About