Recognition of Emotions Using Multichannel EEG Data and DBN-GC-Based Ensemble Deep Learning Framework

Abstract

Fusing multichannel neurophysiological signals to recognize human emotion states becomes increasingly attractive. The conventional methods ignore the complementarity between time domain characteristics, frequency domain characteristics, and time-frequency characteristics of electroencephalogram (EEG) signals and cannot fully capture the correlation information between different channels. In this paper, an integrated deep learning framework based on improved deep belief networks with glia chains (DBN-GCs) is proposed. In the framework, the member DBN-GCs are employed for extracting intermediate representations of EEG raw features from multiple domains separately, as well as mining interchannel correlation information by glia chains. Then, the higher level features describing time domain characteristics, frequency domain characteristics, and time-frequency characteristics are fused by a discriminative restricted Boltzmann machine (RBM) to implement emotion recognition task. Experiments conducted on the DEAP benchmarking dataset achieve averaged accuracy of 75.92% and 76.83% for arousal and valence states classification, respectively. The results show that the proposed framework outperforms most of the above deep classifiers. Thus, potential of the proposed framework is demonstrated.

Electroencephalogram Profiles for Emotion Identification over the Brain Regions Using Spectral, Entropy and Temporal Biomarkers

“…(Age in years, SAQ mean ± standard deviation SD). Respondents rated their responses in the SAQ according to the level of emotion felt, from 5 = very high; 4 = high; 3 = medium; 2 = low, to 1 = very low, thus providing a five-point scale [48]. This enabled the neutral circumstances and six affective states-anger, anxiety, sadness, disgust, surprise and happiness.…”

“…Studies on EEG signal processing have been conducted to identify the brain activity patterns involved in cognitive science, neuropsychological research, clinical assessments, and consciousness research [42][43][44][45][46][47]. Recently, EEG has been widely used to assess and evaluate the human emotional states with excellent time resolution [3,15,[28][29][30]48]. EEG can provide useful information of emotional states that have been described as a potential biomarker to evaluate different emotional responses from multi-channel EEG datasets over the brain regions [38].…”

Deep Learning in Physiological Signal Data: A Survey

“…Hand motion recognition [9][10][11][12][13][14][15][16][17], Muscle activity recognition [18][19][20][21][22][23] ECG Heartbeat signal classification , Heart disease classification [49][50][51][52][53][54][55][56][57][58][59][60][61][62][63], Sleep-stage classification [64][65][66][67][68], Emotion classification [69], age and gender prediction [70] EEG Brain functionality classification , Brain disease classification , Emotion classification [122][123][124][125][126][127][128][129], Sleep-stage classification [130][131][132][133]…”

Low-Rank Deep Convolutional Neural Network for Multitask Learning

“…In recent years, deep learning has been proven to be the most powerful data representation method (Chao, Zhi, Dong and Liu, 2018;Chu, Huang, Xie, Tan, Kamal and Xiong, 2018;Geng, Zhang, Li, Gu, Liang, Liang, Wang, Wu, Patil and Wang, 2017;Glorot, Bordes and Bengio, 2011;Guo, Liu, Oerlemans, Lao, Wu and Lew, 2016;Hu, Wang, Peng, Qiu, Shi and Liu, 2018;Längkvist, Karlsson and Loutfi, 2014;LeCun, Bengio and Hinton, 2015;Ngiam, Khosla, Kim, Nam, Lee and Ng, 2011;Sadouk, Gadi and Essoufi, 2018;Schmidhuber, 2015;Voulodimos, Doulamis, Bebis and Stathaki, 2018a;Voulodimos, Doulamis, Doulamis and Protopapadakis, 2018b;Wu, Zhai, Li, Cui, Wang and Patil;Zhang, Liang, Li, Fang, Wang, Geng and Wang, 2017;Zhang, Liang, Su, Qu and Wang, 2018a). Deep learning methods learn a neural network of multiple layers to extract the hierarchical patterns from the original data, and provide high-level and abstractive features for the learning problems.…”