Spatio-Temporal Analysis of Transformer based Architecture for Attention Estimation from EEG

Delvigne, Victor; Wannous, Hazem; Vandeborre, Jean-Philippe; Ris, Laurence; Dutoit, Thierry

doi:10.1109/icpr56361.2022.9956610

Cited by 5 publications

(3 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By leveraging attention mechanisms, transformer-based methods can achieve performance comparable or even superior to RNN-based approaches [32]. Consequently, researchers have started exploring the application of Transformer-based methods in drowsiness detection, aiming to overcome the limitations associated with long-term sequences often encountered by RNN-based methods [17], [18]. Furthermore, the graph convolutional network (GCN) has become a popular choice for EEG-based drowsiness detection [33].…”

Section: Relatedworkmentioning

confidence: 99%

“…To enhance performance of decoding drowsiness-related brain activities, many researchers have incorporated DL models into their studies. Four main DL models are used in most studies: CNN [13], [14], RNN [15], [16], Transformer [17], [18], and GCN [19], [20]. Although DL-based models have demonstrated significant improvements in decoding drowsy brain activities, most of them do not consider the relative change of brain activities, which is a crucial aspect when dealing with EEG signals.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

SiamEEGNet: Siamese Neural Network-Based EEG Decoding for Drowsiness Detection

Chang,

Chen,

Chang

et al. 2023

Preprint

View full text Add to dashboard Cite

Recent advancements in deep-learning have significantly enhanced EEG-based drowsiness detection. However, most existing methods overlook the importance of relative changes in EEG signals compared to a baseline, a fundamental aspect in conventional EEG analysis including event-related potential and time-frequency spectrograms. We herein introduce SiamEEGNet, a Siamese neural network architecture designed to capture relative changes between EEG data from the baseline and a time window of interest. Our results demonstrate that SiamEEGNet is capable of robustly learning from high-variability data across multiple sessions/subjects and outperforms existing model architectures in cross-subject scenarios. Furthermore, the model's interpretability associates with previous findings of drowsiness-related EEG correlates. The promising performance of SiamEEGNet highlights its potential for practical applications in EEG-based drowsiness detection. We have made the source codes available at http://github.com/CECNL/SiamEEGNet.

show abstract

Section: Relatedworkmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

SiamEEGNet: Siamese Neural Network-Based EEG Decoding for Drowsiness Detection

Chang,

Chen,

Chang

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Xie et al built five Transformer-based deep learning models, and achieved the best accuracy rates of 83.31%, 74.44% and 64.22% in the two-class, three-class and fourclass tasks, respectively [30]. Delvigne et al processed the spatiotemporal information of EEG signals based on the Transformer model and achieved a classification accuracy of more than 74% in the comparison of commonly used models [31].…”

Section: Introductionmentioning

confidence: 99%

Classification of attention deficit/hyperactivity disorder based on EEG signals using a EEG-Transformer model ^∗

He,

Wang,

Yang

et al. 2023

J. Neural Eng.

View full text Add to dashboard Cite

Attention-deficit/hyperactivity disorder (ADHD) is the most common neurodevelopmental disorder in adolescents that can seriously impair a person's attention function, cognitive processes, and learning ability. Currently, clinicians primarily diagnose patients based on the subjective assessments of the Diagnostic and Statistical Manual of Mental Disorders (DMS-5), which can lead to delayed diagnosis of ADHD and even misdiagnosis due to low diagnostic efficiency and lack of well-trained diagnostic experts. Deep learning of electroencephalogram (EEG) signals recorded from ADHD patients could provide an objective and accurate method to assist physicians in clinical diagnosis. This paper proposes the EEG-Transformer deep learning model, which is based on the attention mechanism in the traditional Transformer model, and can perform feature extraction and signal classification processing for the characteristics of EEG signals. A comprehensive comparison was made between the proposed transformer model and three existing convolutional neural network models. The results showed that the proposed EEG-Transformer model achieved an average accuracy of 95.85% and an average AUC value of 0.9926 with the fastest convergence speed, outperforming the other three models. The function and relationship of each module of the module are studied by ablation experiments. The model with optimal performance was identified by the optimization experiment. The EEG-Transformer model proposed in this paper can be used as an auxiliary tool for clinical diagnosis of ADHD, and at the same time provides a basic model for transferable learning in the field of EEG signal classification.

show abstract

Directed Brain Network Transformer for Psychiatric Diagnosis

Zhu,

Qi,

Yue

et al. 2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Spatio-Temporal Analysis of Transformer based Architecture for Attention Estimation from EEG

Cited by 5 publications

References 33 publications

SiamEEGNet: Siamese Neural Network-Based EEG Decoding for Drowsiness Detection

SiamEEGNet: Siamese Neural Network-Based EEG Decoding for Drowsiness Detection

Classification of attention deficit/hyperactivity disorder based on EEG signals using a EEG-Transformer model ^∗

Directed Brain Network Transformer for Psychiatric Diagnosis

Contact Info

Product

Resources

About

Spatio-Temporal Analysis of Transformer based Architecture for Attention Estimation from EEG

Cited by 5 publications

References 33 publications

SiamEEGNet: Siamese Neural Network-Based EEG Decoding for Drowsiness Detection

SiamEEGNet: Siamese Neural Network-Based EEG Decoding for Drowsiness Detection

Classification of attention deficit/hyperactivity disorder based on EEG signals using a EEG-Transformer model ∗

Directed Brain Network Transformer for Psychiatric Diagnosis

Contact Info

Product

Resources

About

Classification of attention deficit/hyperactivity disorder based on EEG signals using a EEG-Transformer model ^∗