Semi-supervised regression with adaptive graph learning for EEG-based emotion recognition

Sha, Tianhui; Zhang, Yikai; Peng, Yong; Kong, Wanzeng

doi:10.3934/mbe.2023505

Cited by 10 publications

(2 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The differential entropy features of each frequency band were then projected onto a two-dimensional matrix ( Li et al, 2018 ; Nguyen et al, 2019 ; Sha et al, 2023 ), with the length and width of the two-dimensional matrix set to H = 9 and W = 9, respectively, and the relative positions of the actual recording electrodes corresponding to the positions of the recording electrodes in the two-dimensional matrix. Figure 4 shows the two-dimensional matrix obtained from the projection based on 32 sampled electrodes, with the unused channel signals filled with zeros.…”

Section: Methodsmentioning

confidence: 99%

Attention-based 3D convolutional recurrent neural network model for multimodal emotion recognition

Du,

Li,

Cheng

et al. 2024

Front. Neurosci.

View full text Add to dashboard Cite

IntroductionMultimodal emotion recognition has become a hot topic in human-computer interaction and intelligent healthcare fields. However, combining information from different human different modalities for emotion computation is still challenging.MethodsIn this paper, we propose a three-dimensional convolutional recurrent neural network model (referred to as 3FACRNN network) based on multimodal fusion and attention mechanism. The 3FACRNN network model consists of a visual network and an EEG network. The visual network is composed of a cascaded convolutional neural network–time convolutional network (CNN-TCN). In the EEG network, the 3D feature building module was added to integrate band information, spatial information and temporal information of the EEG signal, and the band attention and self-attention modules were added to the convolutional recurrent neural network (CRNN). The former explores the effect of different frequency bands on network recognition performance, while the latter is to obtain the intrinsic similarity of different EEG samples.ResultsTo investigate the effect of different frequency bands on the experiment, we obtained the average attention mask for all subjects in different frequency bands. The distribution of the attention masks across the different frequency bands suggests that signals more relevant to human emotions may be active in the high frequency bands γ (31–50 Hz). Finally, we try to use the multi-task loss function Lc to force the approximation of the intermediate feature vectors of the visual and EEG modalities, with the aim of using the knowledge of the visual modalities to improve the performance of the EEG network model. The mean recognition accuracy and standard deviation of the proposed method on the two multimodal sentiment datasets DEAP and MAHNOB-HCI (arousal, valence) were 96.75 ± 1.75, 96.86 ± 1.33; 97.55 ± 1.51, 98.37 ± 1.07, better than those of the state-of-the-art multimodal recognition approaches.DiscussionThe experimental results show that starting from the multimodal information, the facial video frames and electroencephalogram (EEG) signals of the subjects are used as inputs to the emotion recognition network, which can enhance the stability of the emotion network and improve the recognition accuracy of the emotion network. In addition, in future work, we will try to utilize sparse matrix methods and deep convolutional networks to improve the performance of multimodal emotion networks.

show abstract

Section: Methodsmentioning

confidence: 99%

Attention-based 3D convolutional recurrent neural network model for multimodal emotion recognition

Du,

Li,

Cheng

et al. 2024

Front. Neurosci.

View full text Add to dashboard Cite

show abstract

“…Key to developing a pBCI is the ability to reliably detect the different mental states of interest, and research has been done toward detecting such states as fatigue [1,2], attention [3][4][5], stress [6,7], and various emotions [8][9][10][11]. Among the most researched states in pBCI research is mental workload, due to its relevance in many applications, including improving workplace safety in high-risk environments.…”

Section: Introductionmentioning

confidence: 99%

EEG-based hierarchical classification of level of demand and modality of auditory and visual sensory processing

Massaeli,

Power

2024

J. Neural Eng.

View full text Add to dashboard Cite

Objective. To date, most research on EEG-based mental workload detection for passive BCI applications has focused on identifying the overall level of cognitive resources required, such as whether the workload is high or low. We propose, however, that being able to determine the specific type of cognitive resources being used, such as visual or auditory, would also be useful. This would enable the pBCI to take more appropriate action to reduce the overall level of cognitive demand on the user. For example, if a high level of workload was detected and it is determined that the user is primarily engaged in visual information processing, then the pBCI could cause some information to be presented aurally instead. In our previous work we showed that EEG could be used to differentiate visual from auditory processing tasks when the level of processing is high, but the two modalities could not be distinguished when the level of cognitive processing demand was very low. The current study aims to build on this work and move toward the overall objective of developing a pBCI that is capable of predicting both the level and the type of cognitive resources being used. Approach. Fifteen individuals undertook carefully designed visual and auditory tasks while their EEG data was being recorded. In this study, we incorporated a more diverse range of sensory processing conditions including not only single modality conditions (i.e., those requiring one of either visual or auditory processing) as in our previous study, but also dual-modality conditions (i.e., those requiring both visual and auditory processing) and no-task/baseline conditions (i.e., when the individual is not engaged in either visual or auditory processing). Main results. Using regularized linear discriminant analysis within a hierarchical classification algorithm, the overall cognitive demand was predicted with an accuracy of more than 86%, while the presence or absence of visual and auditory sensory processing were each predicted with an accuracy of approximately 70%. Significance. The findings support the feasibility of establishing a pBCI that can determine both the level and type of attentional resources required by the user at any given moment. This pBCI could assist in enhancing safety in hazardous jobs by triggering the most effective and efficient adaptation strategies when high workload conditions are detected.

show abstract

Semi-supervised bipartite graph construction with active EEG sample selection for emotion recognition

Pang,

Peng,

Gao

et al. 2024

Med Biol Eng Comput

View full text Add to dashboard Cite

Semi-supervised regression with adaptive graph learning for EEG-based emotion recognition

Cited by 10 publications

References 32 publications

Attention-based 3D convolutional recurrent neural network model for multimodal emotion recognition

Attention-based 3D convolutional recurrent neural network model for multimodal emotion recognition

EEG-based hierarchical classification of level of demand and modality of auditory and visual sensory processing

Semi-supervised bipartite graph construction with active EEG sample selection for emotion recognition

Contact Info

Product

Resources

About