Phase-Locking Value Based Graph Convolutional Neural Networks for Emotion Recognition

Abstract

Recognition of discriminative neural signatures and regions corresponding to emotions are important in understanding the neuron functional network underlying the human emotion process. Electroencephalogram (EEG) is a spatial discrete signal. In this paper, in order to extract the spatio-temporal characteristics and the inherent information implied by functional connections, a multichannel EEG emotion recognition method based on phase-locking value (PLV) graph convolutional neural networks (P-GCNN) is proposed. The basic idea of the proposed EEG emotion recognition method is using PLV-based brain network to model multi-channel EEG features as graph signals and then perform EEG emotion classification based on this model. Different from the traditional graph convolutional neural networks (GCNN) methods, the proposed P-GCNN method uses the PLV connectivity of EEG signals to determine the mode of emotional-related functional connectivity, which is used to represent the intrinsic relationship between EEG channels in different emotional states. On this basis, the neural network is trained to extract effective EEG emotional features. We conduct extensive experiments on the SJTU emotion EEG dataset (SEED) and DEAP dataset. The experimental results demonstrate that novel framework can improve the classification accuracy on both datasets, but not so effective on DEAP as on SEED, in which with 84.35% classification accuracy for SEED, and the average accuracies of 73.31%, 77.03% and 79.20% are, respectively, obtained for valence, arousal, and dominance classifications on the DEAP database.INDEX TERMS EEG emotion recognition, phase-locking value, graph convolutional neural networks, brain network, functional connectivity.

Video‐triggered EEG‐emotion public databases and current methods: A survey

“…Graph convolutional neural networks (GCNN) is a deep learning network that combines CNN and spectrogram theory to find the relationship within different nodes from graph signals. GCNN is an effective method to extract features from discrete spatial signals [127, 128], which can be used to explore the spatial connection of multi‐dimensional EEG signals for emotion recognition [118]. Jang et al [117] utilized GCNN for emotion recognition based on DEAP database.…”

Frontal EEG Asymmetry of Emotion for the Same Auditory Stimulus

“…Frontal EEG asymmetry refers to the difference in brain activity between the left and right frontal regions [12], [28]. This phenomenon is directly related to emotion and is used to recognize emotion [29]. Positive emotions are specifically associated with left hemisphere activity, whereas negative emotions are associated with more right hemispheric activity [30], [31].…”

A Discriminative Multi-Output Gaussian Processes Scheme for Brain Electrical Activity Analysis

“…Specifically, the coherence (COH), phase value index (PVI), and phase-locking value (PLV) capture pair-wise channel dependencies in the frequency domain. Moreover, the latter two gained attention due to their non-linear capability for unraveling latent connectivity patterns, which have proven valuable for applications, such as motor imagery and emotion recognition [14,15]. Nevertheless, the selection of the connectivity measure is not entirely straightforward.…”