Two-Level Domain Adaptation Neural Network for EEG-Based Emotion Recognition

Abstract

Emotion recognition plays an important part in human-computer interaction (HCI). Currently, the main challenge in electroencephalogram (EEG)-based emotion recognition is the non-stationarity of EEG signals, which causes performance of the trained model decreasing over time. In this paper, we propose a two-level domain adaptation neural network (TDANN) to construct a transfer model for EEG-based emotion recognition. Specifically, deep features from the topological graph, which preserve topological information from EEG signals, are extracted using a deep neural network. These features are then passed through TDANN for two-level domain confusion. The first level uses the maximum mean discrepancy (MMD) to reduce the distribution discrepancy of deep features between source domain and target domain, and the second uses the domain adversarial neural network (DANN) to force the deep features closer to their corresponding class centers. We evaluated the domain-transfer performance of the model on both our self-built data set and the public data set SEED. In the cross-day transfer experiment, the ability to accurately discriminate joy from other emotions was high: sadness (84%), anger (87.04%), and fear (85.32%) on the self-built data set. The accuracy reached 74.93% on the SEED data set. In the cross-subject transfer experiment, the ability to accurately discriminate joy from other emotions was equally high: sadness (83.79%), anger (84.13%), and fear (81.72%) on the self-built data set. The average accuracy reached 87.9% on the SEED data set, which was higher than WGAN-DA. The experimental results demonstrate that the proposed TDANN can effectively handle the domain transfer problem in EEG-based emotion recognition.

Data Augmentation for EEG-Based Emotion Recognition Using Generative Adversarial Networks

“…The linear dynamic system method was used to filter the noises and artifacts that were unrelated to EEG features ( Shi and Lu, 2010 ). Next, the DE features were transformed into 32*32*5 (length and width = 32, channel = 5) topology images according to the method in the literature ( Bao et al, 2021 ), namely TP-DE, as shown in Figure 2 .…”

“…In our previous work ( Bao et al, 2021 ), we proposed a deep neural network (DNN) classification model that can effectively extract the features of the topological graph; its structure is shown in Figure 3 . We add the AdaBN layer ( Li et al, 2018 ) after each convolution layer and full connection layer, which standardizes the distribution between the real samples and the generated samples in each batch.…”

Feature matching as improved transfer learning technique for wearable EEG

“…Common approaches include domainadversarial neural networks [19] and approaches using the maximum mean discrepancy (MMD) loss [20,21]. Unsupervised domain adaptation techniques have already demonstrated their potential for EEG-based applications such as emotion recognition and braincomputer interfaces [22,23,24,25,26]. These domain adaptation techniques have mainly focused on personalization, cross-session adaptation and domain mismatch between different datasets, with the same or very similar channels recorded in both datasets.…”

P3-MSDA: Multi-Source Domain Adaptation Network for Dynamic Visual Target Detection

“…For EEG-based domain adaption, current studies focus on emotion recognition, cognitive load recognition, movement recognition, and motor imagery decoding. Li et al and Bao et al investigated multi-source transfer learning and two-level domain adaptation neural networks, respectively, for cross-subject EEG emotion recognition (Li et al, 2019 ; Bao et al, 2021 ). Jimenez-Guarneros et al proposed custom domain adaptation for cross-subject cognitive load recognition (Jimenez-Guarneros and Gomez-Gil, 2020 ).…”

“…For EEGbased domain adaption, current studies focus on emotion recognition, cognitive load recognition, movement recognition, and motor imagery decoding. Li et al and Bao et al investigated multi-source transfer learning and two-level domain adaptation neural networks, respectively, for cross-subject EEG emotion recognition (Li et al, 2019;Bao et al, 2021 (Tang and Zhang, 2020). However, there is little research on the task of EEG-based target detection, which might be caused by the difficulty of training on an imbalanced dataset of EEG-based target detection.…”