EEG emotion recognition using fusion model of graph convolutional neural networks and LSTM

Yin, Yongqiang; Zheng, Xiangwei; Hu, Bin; Zhang, Yuang; Cui, Xinchun

doi:10.1016/j.asoc.2020.106954

“…Differential entropy was extracted to construct a feature cube as the input of the model. The average classification accuracies were 90.45% and 90.60% for valence and arousal on the DEAP dataset [ 27 ]. A dynamical graph convolutional neural network (DGCNN) using a graph to model the multichannel EEG features was proposed in [ 28 ].…”

Section: Related Workmentioning

confidence: 99%

EEG-Based Emotion Recognition Using an Improved Weighted Horizontal Visibility Graph

Kong

¹

,

Shao

²

,

Hu

³

et al. 2021

Sensors

View full text Add to dashboard Cite

Emotion recognition, as a challenging and active research area, has received considerable awareness in recent years. In this study, an attempt was made to extract complex network features from electroencephalogram (EEG) signals for emotion recognition. We proposed a novel method of constructing forward weighted horizontal visibility graphs (FWHVG) and backward weighted horizontal visibility graphs (BWHVG) based on angle measurement. The two types of complex networks were used to extract network features. Then, the two feature matrices were fused into a single feature matrix to classify EEG signals. The average emotion recognition accuracies based on complex network features of proposed method in the valence and arousal dimension were 97.53% and 97.75%. The proposed method achieved classification accuracies of 98.12% and 98.06% for valence and arousal when combined with time-domain features.

show abstract

“…Some research work uses CNNs to online characters after converting the handwriting trajectory to image-like representations. Also, Yuan et al [25] proposed word-level recognition for Latin script by employing CNN architecture. In this article, CNN is used for online handwritten document recognition after online word segmentation.…”

Section: Deep Learning-based Approachesmentioning

confidence: 99%

“…Further, several researches focus specifically on mono-language script, for instance, Arabic [13], Chinese [33], Tamil [34], Japanese [35] etc. Similarly, there exist multi-script and multi-language systems for online handwriting recognition like those presented in [35], [21] as well as other commercial systems such as those developed by Apple [25] and Microsoft [36].…”

Section: Deep Learning-based Approachesmentioning

confidence: 99%

Hybrid Architecture based on RNN-SVM for Multilingual Handwriting Recognition using Beta-elliptic and CNN models

Hamdi¹,

Boubaker²,

Rabhi³

et al. 2021

Preprint

2

0

View full text Add to dashboard Cite

Currently, deep learning approaches have proven successful in the areas of handwriting recognition. Despite this, research in this field is still needed, especially in the context of multilingual online handwriting recognition scripts by adopting new network architectures and combining relevant parametric models. In this paper, we propose a multi-stage deep learning-based algorithm for multilingual online handwriting recognition based on hybrid deep Bidirectional Long Short Term Memory (DBLSTM) and SVM networks. The main contributions of our work lie in partly in the composition of a new multi-stage architecture of deep learning networks associated with effective feature vectors that integrate dynamic and visual characteristics. First, the proposed system proceeds by pretreating the acquired script and delimiting its Segments of Online Handwriting Trajectories (SOHTs). Second, two types of feature vectors combining BetaElliptic Model (BEM) and Convolutional Neural Network (CNN) are extracted for each SOHT in order to fuzzy classify them into k sub-groups using DBLSTM neural networks for both online and offline branches trained using an unsupervised fuzzy k-means algorithm. Finally, we combine the trained models to strengthen the discrimination power of the global system using SVM engine. Extensive experiments on three data sets were conducted to validate the performance of the proposed method. The experimental results show the effectiveness and complementarities of the individual modules and the advantage of their fusion.<br>

show abstract

“…Some research work uses CNNs to online characters after converting the handwriting trajectory to image-like representations. Also, Yuan et al [52] proposed word-level recognition for Latin script by employing CNN architecture. In this article, CNN is used for online handwritten document recognition after online word segmentation.…”

Section: Deep Learning-based Approachesmentioning

confidence: 99%

“…RNN with BLSTM (e.g., [51]) and BLSTM with gated recurrent unit (GRU) for online Chinese characters recognition and generation [57]. More recently, a combination of graph CNN and LSTM is introduced also by [52] for EEG emotion recognition.…”

Section: Deep Learning-based Approachesmentioning

confidence: 99%

Hybrid Architecture based on RNN-SVM for Multilingual Handwriting Recognition using Beta-elliptic and CNN models

Hamdi¹,

Boubaker²,

Rabhi³

et al. 2021

Preprint

1

0

View full text Add to dashboard Cite

Currently, deep learning approaches have proven successful in the areas of handwriting recognition. Despite this, research in this field is still needed, especially in the context of multilingual online handwriting recognition scripts by adopting new network architectures and combining relevant parametric models. In this paper, we propose a multi-stage deep learning-based algorithm for multilingual online handwriting recognition based on hybrid deep Bidirectional Long Short Term Memory (DBLSTM) and SVM networks. The main contributions of our work lie in partly in the composition of a new multi-stage architecture of deep learning networks associated with effective feature vectors that integrate dynamic and visual characteristics. First, the proposed system proceeds by pretreating the acquired script and delimiting its Segments of Online Handwriting Trajectories (SOHTs). Second, two types of feature vectors combining BetaElliptic Model (BEM) and Convolutional Neural Network (CNN) are extracted for each SOHT in order to fuzzy classify them into k sub-groups using DBLSTM neural networks for both online and offline branches trained using an unsupervised fuzzy k-means algorithm. Finally, we combine the trained models to strengthen the discrimination power of the global system using SVM engine. Extensive experiments on three data sets were conducted to validate the performance of the proposed method. The experimental results show the effectiveness and complementarities of the individual modules and the advantage of their fusion.<br>

show abstract

EEG emotion recognition using fusion model of graph convolutional neural networks and LSTM

Cited by 316 publications

References 34 publications

EEG-Based Emotion Recognition Using an Improved Weighted Horizontal Visibility Graph

EEG-Based Emotion Recognition Using an Improved Weighted Horizontal Visibility Graph

Hybrid Architecture based on RNN-SVM for Multilingual Handwriting Recognition using Beta-elliptic and CNN models

Hybrid Architecture based on RNN-SVM for Multilingual Handwriting Recognition using Beta-elliptic and CNN models

Contact Info

Product

Resources

About