Decoupling representation learning for imbalanced electroencephalography classification in rapid serial visual presentation task

Abstract

Objective. The class imbalance problem considerably restricts the performance of electroencephalography (EEG) classification in the rapid serial visual presentation (RSVP) task. Existing solutions typically employ re-balancing strategies (e.g. re-weighting and re-sampling) to alleviate the impact of class imbalance, which enhances the classifier learning of deep networks but unexpectedly damages the representative ability of the learned deep features as original distributions become distorted. Approach. In this study, a novel decoupling representation learning (DRL) model, has been proposed that separates the representation learning and classification processes to capture the discriminative feature of imbalanced RSVP EEG data while classifying it accurately. The representation learning process is responsible for learning universal patterns for the classification of all samples, while the classifier determines a better bounding for the target and non-target classes. Specifically, the representation learning process adopts a dual-branch architecture, which minimizes the contrastive loss to regularize the representation space. In addition, to learn more discriminative information from RSVP EEG data, a novel multi-granular information (MGI) based extractor is designed to extract spatial-temporal information. Considering the class re-balancing strategies can significantly promote classifier learning, the classifier was trained with rebalanced EEG data while freezing the parameters of the representation learning process. Main results. To evaluate the proposed method, experiments were conducted on two public datasets and one self-conducted dataset. The results demonstrate that the proposed DRL can achieve state-of-the-art performance for EEG classification in the RSVP task. Significance. This is the first study to focus on the class imbalance problem and propose a generic solution in the RSVP task. Furthermore, multi-granular data was explored to extract more complementary spatial-temporal information. The code is open-source and available at https://github.com/Tammie-Li/DRL.

Multi-Task Collaborative Network: Bridge the Supervised and Self-Supervised Learning for EEG Classification in RSVP Tasks

“…1) Representative method to learn more discriminative representation in the presence of signal-to-noise ratio RSVP EEG data: There are four traditional SL methods and six deep learning methods, which have been commonly used for RSVP EEG classification. Traditional SL methods include rLDA [23], HDCA [29], xDAWN-RG [30], [31], XGB-DIM [32], and deep learning methods consist of DeepConvNet [33], EEGNet [24], EEG-Inception [34], PLNet [19], PPNN [10], and DRL [20]. 2) Representative method to alleviate the negative impact of noisy labels: We select five methods, e.g.…”

“…Considering the phase-locked characteristics in event-related potential (ERP) components, Zang et al [19] and Li et al [10] proposed PLNet and PPNN to improve classification performance by capturing phase information from RSVP EEG data. Later, in another study, Li et al [20] further considered the class imbalance problem of RSVP tasks and proposed a DRL model to alleviate the negative impact.…”

“…Recent deep-learning literature has also discussed this issue. For example, PPNN [10] and PLNet [19] have been proposed to take the phase characteristic of RSVP EEG signals into consideration, and DRL [20] has been proposed to alleviate the class imbalance between targets and non-targets. Although these deep learning methods can capture certain discriminative information from RSVP EEG signals, they do not sufficiently leverage the prior ERP-related information.…”

A review of deep learning methods for cross-subject rapid serial visual presentation detection in World Robot Contest 2022

“…In the competition, unweighted average recall (UAR) [24] was used as the scoring criterion, where n denotes the total number of categories, w r denotes the weight factor applied for each category, which is set to [0 33,. 0.33, 0.33], t r denotes the number of images per category, and c r was the number of correct predictions per category.…”

“…In the competition, unweighted average recall (UAR) [24] was used as the scoring criterion, Brain Sci. Adv.…”