A GAN-Based Data Augmentation Method for Multimodal Emotion Recognition

Luo, Yaping; Zhu, Li-Zhen; Lu, Bao-Liang

doi:10.1007/978-3-030-22796-8_16

Cited by 23 publications

(12 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The positive experiment results on SEED and DEAP emotion recognition datasets proved the effectiveness of the CWGAN model. A conditional boundary equilibrium GAN-based EEG data augmentation method [126] for artificial differential entropy features generation was also proven to be effective in improving multimodal emotion recognition performance.…”

Section: Eeg Data Augmentationmentioning

confidence: 99%

EEG-Based Brain-Computer Interfaces (BCIs): A Survey of Recent Studies on Signal Sensing Technologies and Computational Intelligence Approaches and Their Applications

Cao

Jolfaei

et al. 2021

IEEE/ACM Trans. Comput. Biol. and Bioinf.

248

View full text Add to dashboard Cite

Brain-Computer interfaces (BCIs) enhance the capability of human brain activities to interact with the environment. Recent advancements in technology and machine learning algorithms have increased interest in electroencephalographic (EEG)-based BCI applications. EEG-based intelligent BCI systems can facilitate continuous monitoring of fluctuations in human cognitive states under monotonous tasks, which is both beneficial for people in need of healthcare support and general researchers in different domain areas. In this review, we survey the recent literature on EEG signal sensing technologies and computational intelligence approaches in BCI applications, compensating for the gaps in the systematic summary of the past five years. Specifically, we first review the current status of BCI and signal sensing technologies for collecting reliable EEG signals. Then, we demonstrate state-of-the-art computational intelligence techniques, including fuzzy models and transfer learning in machine learning and deep learning algorithms, to detect, monitor, and maintain human cognitive states and task performance in prevalent applications. Finally, we present a couple of innovative BCI-inspired healthcare applications and discuss future research directions in EEG-based BCI research.

show abstract

Section: Eeg Data Augmentationmentioning

confidence: 99%

EEG-Based Brain-Computer Interfaces (BCIs): A Survey of Recent Studies on Signal Sensing Technologies and Computational Intelligence Approaches and Their Applications

Cao

Jolfaei

et al. 2021

IEEE/ACM Trans. Comput. Biol. and Bioinf.

248

View full text Add to dashboard Cite

show abstract

“…For the SEED dataset, since there are three classes, we are assuming that the chance level accuracy is 33.33%. For DEAP dataset, the chance level accuracy is 50% for binary classification.In 2019, Luo et al adopted a conditional Boundary Equilibrium GAN (cBEGAN) to generate artificial differential entropy features of EEG signals on 2 popular emotion recognition dataset (SEED, SEED V)[72]. cBEGAN used the Wasserstein distance to measure the difference between two reconstruction loss distributions.…”

mentioning

confidence: 99%

Data augmentation for deep-learning-based electroencephalography

Lashgari

Liang

Maoz

2020

Journal of Neuroscience Methods

273

140

View full text Add to dashboard Cite

Highlights • Data augmentation (DA) is increasingly used with deep learning (DL) on EEG • It enhances decoding accuracy left unexplained by 29% on average on the datasets we review • We analyze which specific DA techniques appear to work best for which EEG tasks • We tested various DA techniques on an open motor-imagery task and compared the accuracy gains to demonstrate the usefulness of DA for DL-based EEG analysis • We propose guidelines for reporting parameters for different DA techniques Abstract-Background Data augmentation (DA) has recently been demonstrated to achieve considerable performance gains for deep learning (DL)-increased accuracy and stability and reduced overfitting. Some electroencephalography (EEG) tasks suffer from low samples-to-features ratio, severely reducing DL effectiveness. DA with DL thus holds transformative promise for EEG processing, possibly like DL revolutionized computer vision, etc.-New method We review trends and approaches to DA for DL in EEG to address: Which DA approaches exist and are common for which EEG tasks? What input features are used? And, what kind of accuracy gain can be expected?-Results DA for DL on EEG begun 5 years ago and is steadily used more. We grouped DA techniques (noise addition, generative adversarial networks, sliding windows, sampling, Fourier transform, recombination of segmentation, and others) and EEG tasks (into seizure detection, sleep stages, motor imagery, mental workload, emotion recognition, motor tasks, and visual tasks). DA efficacy across techniques varied considerably. Noise addition and sliding windows provided the highest accuracy boost; mental workload most benefitted from DA. Sliding window, noise addition, and sampling methods most common for seizure detection, mental workload, and sleep stages, respectively.-Comparing with existing methods Percent of decoding accuracy explained by DA beyond unaugmented accuracy varied between 8% for recombination of segmentation and 36% for noise addition and from 14% for motor imagery to 56% for mental workload-29% on average.-Conclusions DA increasingly used and considerably improved DL decoding accuracy on EEG. Additional publications-if adhering to our reporting guidelines-will facilitate more detailed analysis.

show abstract

“…Recently, Generative Adversarial Networks (GANs) have revealed their potential in generating EEG signals that mimic real ones, utilized in the emotion recognition task [21]- [23] and a wide variety of applications [24]- [28]. A conditional version of the Wasserstein Generative Adversarial Network (WGAN) was used to augment EEG data for emotion recognition in [22].…”

Section: B Data Augmentation For Eeg-based Emotion Recognitionmentioning

confidence: 99%

“…Luo et al proposed to use a conditional Boundary Equilibrium GAN (cBEGAN) to generate artificial differential entropy features of original EEG data, eye movement data and their concatenations for multi-modal emotion recognition. The main advantage of it is that the proposed GAN has good stability and a very quick convergence speed [23]. Luo et al proposed three methods for augmenting EEG training data to enhance the performance of emotion recognition models, including conditional Wasserstein GAN, selective variational autoencoder, and selective WGAN [21].…”

Section: B Data Augmentation For Eeg-based Emotion Recognitionmentioning

confidence: 99%

GANSER: A Self-supervised Data Augmentation Framework for EEG-based Emotion Recognition

Zhang¹,

Zhong²,

Liu³

2021

Preprint

View full text Add to dashboard Cite

The data scarcity problem in Electroencephalography (EEG) based affective computing results into difficulty in building an effective model with high accuracy and stability using machine learning algorithms especially deep learning models. Data augmentation has recently achieved considerable performance improvement for deep learning models-increased accuracy, stability, and reduced over-fitting. In this paper, we propose a novel data augmentation framework, namely Generative Adversarial Network-based Self-supervised Data Augmentation (GANSER). As the first to combine adversarial training with self-supervised learning for EEG-based emotion recognition, the proposed framework can generate high-quality and high-diversity simulated EEG samples. In particular, we utilize adversarial training to learn an EEG generator and force the generated EEG signals to approximate the distribution of real samples, ensuring the quality of augmented samples. A transformation function is employed to mask parts of EEG signals and force the generator to synthesize potential EEG signals based on the remaining parts, to produce a wide variety of samples. The masking possibility during transformation is introduced as prior knowledge to guide to extract distinguishable features for simulated EEG signals and generalize the classifier to the augmented sample space. Finally, extensive experiments demonstrate our proposed method can help emotion recognition for performance gain and achieve stateof-the-art results.

show abstract

A GAN-Based Data Augmentation Method for Multimodal Emotion Recognition

Cited by 23 publications

References 10 publications

EEG-Based Brain-Computer Interfaces (BCIs): A Survey of Recent Studies on Signal Sensing Technologies and Computational Intelligence Approaches and Their Applications

EEG-Based Brain-Computer Interfaces (BCIs): A Survey of Recent Studies on Signal Sensing Technologies and Computational Intelligence Approaches and Their Applications

Data augmentation for deep-learning-based electroencephalography

GANSER: A Self-supervised Data Augmentation Framework for EEG-based Emotion Recognition

Contact Info

Product

Resources

About