Feature Pyramid Networks and Long Short-Term Memory for EEG Feature Map-Based Emotion Recognition

Abstract

The original EEG data collected are the 1D sequence, which ignores spatial topology information; Feature Pyramid Networks (FPN) is better at small dimension target detection and insufficient feature extraction in the scale transformation than CNN. We propose a method of FPN and Long Short-Term Memory (FPN-LSTM) for EEG feature map-based emotion recognition. According to the spatial arrangement of brain electrodes, the Azimuth Equidistant Projection (AEP) is employed to generate the 2D EEG map, which preserves the spatial topology information; then, the average power, variance power, and standard deviation power of three frequency bands (α, β, and γ) are extracted as the feature data for the EEG feature map. BiCubic interpolation is employed to interpolate the blank pixel among the electrodes; the three frequency bands EEG feature maps are used as the G, R, and B channels to generate EEG feature maps. Then, we put forward the idea of distributing the weight proportion for channels, assign large weight to strong emotion correlation channels (AF3, F3, F7, FC5, and T7), and assign small weight to the others; the proposed FPN-LSTM is used on EEG feature maps for emotion recognition. The experiment results show that the proposed method can achieve Value and Arousal recognition rates of 90.05% and 90.84%, respectively.

EEG-Based Emotion Recognition with Consideration of Individual Difference

“…Zhu et al [ 18 ] proposed an emotion recognition method considering multi-band EEG data inputs based on a dynamic Simplified Graph Convolution (SGC) network and a channel-style recalibration module. Zhang et al [ 19 ] proposed the idea of assigning channel weight ratios to the channels that are more strongly correlated with emotion. By using strong emotion correlation channels to assign large weights, their method was able to achieve recognition rates of 90.05% and 90.84%, respectively, in terms of potency and arousal.…”

EEG Emotion Recognition by Fusion of Multi-Scale Features

“…It constructs a feature pyramid with different resolutions and utilizes both top-down and bottom-up pathways to integrate features, thereby enhancing feature representation. FPN has been successfully applied in EEG signal research and achieved good classification performance [ 21 ]. On the other hand, Deformable Convolution (DCN) [ 22 , 23 ] is a type of convolution operation with deformation-awareness.…”