Comparison of Attention-based Deep Learning Models for EEG Classification

Abstract

To evaluate the impact on Electroencephalography (EEG) classification of different kinds of attention mechanisms in Deep Learning (DL) models. Methods: We compared three attention-enhanced DL models, the brand-new InstaGATs, an LSTM with attention and a CNN with attention. We used these models to classify normal and abnormal (i.e., artifactual or pathological) EEG patterns. Results: We achieved the state of the art in all classification problems, regardless the large variability of the datasets and the simple architecture of the attentionenhanced models. We could also proved that, depending on how the attention mechanism is applied and where the attention layer is located in the model, we can alternatively leverage the information contained in the time, frequency or space domain of the dataset. Conclusions: with this work, we shed light over the role of different attention mechanisms in the classification of normal and abnormal EEG patterns. Moreover, we discussed how they can exploit the intrinsic relationships in the temporal, frequency and spatial domains of our brain activity. Significance: Attention represents a promising strategy to evaluate the quality of the EEG information, and its relevance, in different real-world scenarios. Moreover, it can make it easier to parallelize the computation and, thus, to speed up the analysis of big electrophysiological (e.g., EEG) datasets.

EEG-ConvTransformer for Single-Trial EEG based Visual Stimuli Classification

“…While most of the works on transformers reverberated the temporal domain, employing the multi-head attention over the spatial ordinate to encompass the inter-region representational similarities may be more beneficial. This idea is further endorsed by literature on other EEG signal classification systems, where the brain connectivity and inter-channel relationships are hinted to be beneficial [22,19].…”

“…The deep learning architectures, although newly introduced, have superseded their performances [13,14,15]. Convolutional Neural Networks (CNNs), Deep Belief Networks (DBNs), and Recurrent Neural Networks (RNNs) are the primarily used architectures [16,17], with sparing, but increasing involvement of attention-based models [18,19]. The Transformer Network [20] that employs Multi-Head Attention (MHA), has recently been introduced to EEG systems [21].…”

A channel-wise attention-based representation learning method for epileptic seizure detection and type classification

“…In work [17], the author has discussed the difference be-tween machine learning techniques and deep learning method in distinguishing patients under antiepileptic drugs and those taking no medications, as well as between the two anticonvulsants. The method was validated on TUSZ dataset since it is the largest available dataset [5] The comparison invoked in the work [18] shows that a small difference exists between the used ML techniques and deep model in achieving a moderate accuracy rate for medication use detection.…”

ACTA: A Mobile-Health Solution for Integrated Nudge-Neurofeedback Training for Senior Citizens

“…Similarly, if we assume that no attentional state is required during the journey between two consecutive landmarks (i.e., passing by the non-relevant places), then a negative label can be associated to the corresponding EEG data. Thus, we could exploit advanced supervised ML methods to train a binary classifier that could accurately classify the EEG data either as attention-related or non-attention related samples [67,68,69,70,71]. At this stage, several deep architectures can be used to identify the brain networks features which are related to the individual's attentional state.…”

“…At this stage, several deep architectures can be used to identify the brain networks features which are related to the individual's attentional state. Moreover, by designing and implementating specific "attentional mechanisms", we could weight those pieces of EEG information that mostly impact on determining the classification of the individual's attentional level [67,68].…”