EEG2Vec: Learning Affective EEG Representations via Variational Autoencoders

Bethge, David; Hallgarten, Philipp; Große-Puppendahl, Tobias; Mohamed, Kari,; Chuang, Lewis L.; Özdenizci, Ozan; Schmidt, Albrecht

doi:10.1109/smc53654.2022.9945517

Cited by 21 publications

(3 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[25] explored traditional AE and variational AE for EEG emotion recognition with the latent features learned by the AEs. [47] proposed a conditional variational AE to perform emotion recognition with the compressed latent representation, where the proposed method conditions the decoder on the class label and participant ID. In this work, we aim to explore the usage of AE by learning a latent representation of the EEG channel dimension.…”

Section: A Autoencodersmentioning

confidence: 99%

TASA: Temporal Attention With Spatial Autoencoder Network for Odor-Induced Emotion Classification Using EEG

Tong,

Ding,

Zhang

et al. 2024

IEEE Trans. Neural Syst. Rehabil. Eng.

View full text Add to dashboard Cite

The olfactory system enables humans to smell different odors, which are closely related to emotions. The high temporal resolution and non-invasiveness of Electroencephalogram (EEG) make it suitable to objectively study human preferences for odors. Effectively learning the temporal dynamics and spatial information from EEG is crucial for detecting odor-induced emotional valence. In this paper, we propose a deep learning architecture called Temporal Attention with Spatial Autoencoder Network (TASA) for predicting odor-induced emotions using EEG. TASA consists of a filter-bank layer, a spatial encoder, a time segmentation layer, a Long Short-Term Memory (LSTM) module, a multi-head self-attention (MSA) layer, and a fully connected layer. We improved upon the previous work by utilizing a two-phase learning framework, using the autoencoder module to learn the spatial information among electrodes by reconstructing the given input with a latent representation in the spatial dimension, which aims to minimize information loss compared to spatial filtering with CNN. The second improvement is inspired by the continuous nature of the olfactory process; we propose to use LSTM-MSA in TASA to capture its temporal dynamics by learning the intercorrelation among the time segments of the EEG. TASA is evaluated on an existing olfactory EEG dataset and compared with several existing deep learning architectures to demonstrate its effectiveness in predicting olfactory-triggered emotional responses. Interpretability analyses with DeepLIFT also suggest that TASA learns spatial-spectral features that are relevant to olfactory-induced emotion recognition.

show abstract

Section: A Autoencodersmentioning

confidence: 99%

TASA: Temporal Attention With Spatial Autoencoder Network for Odor-Induced Emotion Classification Using EEG

Tong,

Ding,

Zhang

et al. 2024

IEEE Trans. Neural Syst. Rehabil. Eng.

View full text Add to dashboard Cite

show abstract

“…To achieve further improvement and learn more general features that can reveal or separate different factors of the phenomena entangled in the input data, unsupervised learning methods are introduced to make use of knowledge learned from a different EEG task [54]- [58]. For instance, autoencoders are first trained to reconstruct EEG time series before fine-tuning the encoder to a classification task [59]- [63]. These methods indicate that downstream EEG tasks can also benefit from more general feature extractors to a certain extent.…”

Section: A Pretrained Models For Eeg Analysismentioning

confidence: 99%

Speech2EEG: Leveraging Pretrained Speech Model for EEG Signal Recognition

Zhou

Duan

Zou

et al. 2023

IEEE Trans. Neural Syst. Rehabil. Eng.

View full text Add to dashboard Cite

Identifying meaningful brain activities is critical in brain-computer interface (BCI) applications. Recently, an increasing number of neural network approaches have been proposed to recognize EEG signals. However, these approaches depend heavily on using complex network structures to improve the performance of EEG recognition and suffer from the deficit of training data. Inspired by the waveform characteristics and processing methods shared between EEG and speech signals, we propose Speech2EEG, a novel EEG recognition method that leverages pretrained speech features to improve the accuracy of EEG recognition. Specifically, a pretrained speech processing model is adapted to the EEG domain to extract multichannel temporal embeddings. Then, several aggregation methods, including the weighted average, channelwise aggregation, and channel-and-depthwise aggregation, are implemented to exploit and integrate the multichannel temporal embeddings. Finally, a classification network is used to predict EEG categories based on the integrated features. Our work is the first to explore the use of pretrained speech models for EEG signal analysis as well as the effective ways to integrate the multichannel temporal embeddings from the EEG signal. Extensive experimental results suggest that the proposed Speech2EEG method achieves stateof-the-art performance on two challenging motor imagery (MI) datasets, the BCI IV-2a and BCI IV-2b datasets, with accuracies of 89.5% and 84.07%, respectively. Visualization analysis of the multichannel temporal embeddings show that the Speech2EEG architecture can capture useful patterns related to MI categories, which can provide a novel solution for subsequent research under the constraints of a limited dataset scale.

show abstract

“…The drawback of auto-encoder is its strong tendency to over-fit [26], as it is solely trained to encode and decode with as little loss as possible regardless of how the latent space is organized [32], VAE has been developed as an effective solutions [26,2], e.g. VAEs has been used in EEG classification tasks to learn robust features [33,1,2,3].…”

Section: Related Work 1vaementioning

confidence: 99%