EEG-based emotion recognition using domain adaptation network

Jin, Yi-Ming; Luo, Yudong; Zheng, Wei‐Long; Lu, Bao-Liang

doi:10.1109/icot.2017.8336126

Cited by 45 publications

(14 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This disturbance causes the baseline EEG signals to be unable to characterize the differences in participant characteristics found in the EEG signals. There are several methods applicable to eliminate disturbance/artifacts in the EEG signals, including regression [50], wavelet transform [51], and blind source separation (BSS), which further include other techniques such as independent component analysis (ICA) usually applied for electrooculography (EOG) artifacts [5], [29] and eye blinking [52]. This ICA also has the ability to remove artifacts using statistical independence between EEG and artifacts [10].…”

Section: Rq 2: How Can An Eeg Signal Be Generated With Consideration Of Differences In Participant Characteristics?mentioning

confidence: 99%

The challenges of emotion recognition methods based on electroencephalogram signals: a literature review

Wirawan

Wardoyo

Lelono

2022

IJECE

View full text Add to dashboard Cite

Electroencephalogram (EEG) signals in recognizing emotions have several advantages. Still, the success of this study, however, is strongly influenced by: i) the distribution of the data used, ii) consider of differences in participant characteristics, and iii) consider the characteristics of the EEG signals. In response to these issues, this study will examine three important points that affect the success of emotion recognition packaged in several research questions: i) What factors need to be considered to generate and distribute EEG data?, ii) How can EEG signals be generated with consideration of differences in participant characteristics?, and iii) How do EEG signals with characteristics exist among its features for emotion recognition? The results, therefore, indicate some important challenges to be studied further in EEG signals-based emotion recognition research. These include i) determine robust methods for imbalanced EEG signals data, ii) determine the appropriate smoothing method to eliminate disturbances on the baseline signals, iii) determine the best baseline reduction methods to reduce the differences in the characteristics of the participants on the EEG signals, iv) determine the robust architecture of the capsule network method to overcome the loss of knowledge information and apply it in more diverse data set.

show abstract

Section: Rq 2: How Can An Eeg Signal Be Generated With Consideration Of Differences In Participant Characteristics?mentioning

confidence: 99%

The challenges of emotion recognition methods based on electroencephalogram signals: a literature review

Wirawan

Wardoyo

Lelono

2022

IJECE

View full text Add to dashboard Cite

show abstract

“…In recent years, with the development of deep learning techniques and its usability, many works of EEG-based decoding with neural networks have been proposed. Jin et al ( 2017 ) and Li et al ( 2018 ) adopts deep adaptation network (DAN) (Long et al, 2015 ) to EEG-based emotion recognition, which takes maximum mean discrepancy (MMD) (Borgwardt et al, 2006 ) as a measure of the distance between the source and the target domain, and training to reduce it on multiple layers. Extending the original method, Chai et al proposes subspace alignment auto-encoder (SAAE) (Chai et al, 2016 ) which first projects both source and target domains into a domain-invariant subspace using an auto-encoder, and then kernel PCA, graph regularization and MMD are used to align the feature distribution.…”

Section: Related Workmentioning

confidence: 99%

“…Due to the shift between domains, adopting DA for EEG data especially when facing multiple sources is difficult. In recent years, the researchers tend to merge all source domains into one single source and then use DA to align the distribution (Source-combine DA in Figure 1 ) (Zheng and Lu, 2016 ; Jin et al, 2017 ; Li et al, 2018 , 2019a , b , 2020 ; Zheng et al, 2018 ; Zhao et al, 2021 ). This simple approach may improve the performance because it expands the training data for the model, but it ignores the non-stationary of each EEG source domain itself and disrupts it (i.e., EEG data of different people obey different marginal distributions), besides, directly merging into one new source domain cannot determine whether its new marginal distribution still obeys EEG-data distribution, thus brings a larger bias.…”

Section: Introductionmentioning

confidence: 99%

MS-MDA: Multisource Marginal Distribution Adaptation for Cross-Subject and Cross-Session EEG Emotion Recognition

Chen

Jin

et al. 2021

Front. Neurosci.

View full text Add to dashboard Cite

As an essential element for the diagnosis and rehabilitation of psychiatric disorders, the electroencephalogram (EEG) based emotion recognition has achieved significant progress due to its high precision and reliability. However, one obstacle to practicality lies in the variability between subjects and sessions. Although several studies have adopted domain adaptation (DA) approaches to tackle this problem, most of them treat multiple EEG data from different subjects and sessions together as a single source domain for transfer, which either fails to satisfy the assumption of domain adaptation that the source has a certain marginal distribution, or increases the difficulty of adaptation. We therefore propose the multi-source marginal distribution adaptation (MS-MDA) for EEG emotion recognition, which takes both domain-invariant and domain-specific features into consideration. First, we assume that different EEG data share the same low-level features, then we construct independent branches for multiple EEG data source domains to adopt one-to-one domain adaptation and extract domain-specific features. Finally, the inference is made by multiple branches. We evaluate our method on SEED and SEED-IV for recognizing three and four emotions, respectively. Experimental results show that the MS-MDA outperforms the comparison methods and state-of-the-art models in cross-session and cross-subject transfer scenarios in our settings. Codes at https://github.com/VoiceBeer/MS-MDA.

show abstract

“…In recent years, with the development of deep learning techniques and its usability, many works of EEG-based decoding with neural networks have been proposed. Jin et al [27], and Li et al [28] adopts deep adaptation network (DAN) [29] to EEG-based emotion recognition, which takes maximum mean discrepancy (MMD) [30] as a measure of the distance between the source and the target domain, and training to reduce it on multiple layers. Extending the original method, Chai et al proposes subspace alignment auto-encoder (SAAE) [31] which first projects both source and target domains into a domain-invariant subspace using an auto-encoder, and then kernel PCA, graph regularization and MMD are used to align the feature distribution.…”

Section: Related Workmentioning

confidence: 99%

MS-MDA: Multisource Marginal Distribution Adaptation for Cross-subject and Cross-session EEG Emotion Recognition

Chen¹,

Jin²,

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

As an essential element for the diagnosis and rehabilitation of psychiatric disorders, the electroencephalogram (EEG) based emotion recognition has achieved significant progress due to its high precision and reliability. However, one obstacle to practicality lies in the variability between subjects and sessions. Although several studies have adopted domain adaptation (DA) approaches to tackle this problem, most of them treat multiple EEG data from different subjects and sessions together as a single source domain for transfer, which either fails to satisfy the assumption of domain adaptation that the source has a certain marginal distribution, or increases the difficulty of adaptation. We therefore propose the multi-source marginal distribution adaptation (MS-MDA) for EEG emotion recognition, which takes both domain-invariant and domain-specific features into consideration. First, we assume that different EEG data share the same low-level features, then we construct independent branches for multiple EEG data source domains to adopt oneto-one domain adaptation and extract domain-specific features. Finally, the inference is made by multiple branches. We evaluate our method on SEED and SEED-IV for recognizing three and four emotions, respectively. Experimental results show that the MS-MDA outperforms the comparison methods and state-of-theart models in cross-session and cross-subject transfer scenarios in our settings. Codes at https://github.com/VoiceBeer/MS-MDA.

show abstract

EEG-based emotion recognition using domain adaptation network

Cited by 45 publications

References 11 publications

The challenges of emotion recognition methods based on electroencephalogram signals: a literature review

The challenges of emotion recognition methods based on electroencephalogram signals: a literature review

MS-MDA: Multisource Marginal Distribution Adaptation for Cross-Subject and Cross-Session EEG Emotion Recognition

MS-MDA: Multisource Marginal Distribution Adaptation for Cross-subject and Cross-session EEG Emotion Recognition

Contact Info

Product

Resources

About