A channel-mixing convolutional neural network for motor imagery EEG decoding and feature visualization

Ma, Weifeng; Gong, Yifei; Zhou, Gongxue; Liu, Yang; Zhang, Lei; He, Boxian

doi:10.1016/j.bspc.2021.103021

Cited by 19 publications

(9 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…That is, fixing the sliding short-time window length parameter τ =2 s with an overlapping step of 1 s, resulting in N τ =5 EEG segments. For implementing the filter bank strategy, the following bandwidths of interest: ∆ f ∈{µ∈ [8][9][10][11][12], β∈ [12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28][29][30]} Hz. These bandwidths belong to µ, and β rhythms, commonly associated with electrical brain activities provoked by MI tasks [53].…”

Section: Preprocessing and Feature Extraction Of Image-based Represen...mentioning

confidence: 99%

“…Since the way the convolution kernels learn features within CNN frameworks directly influences final performance outcomes, visualizing the inputs that mainly excite the individual activation patterns of weights learned at any layer of the model can aid interpretation [28]. Several approaches to analyzing EEG decoding models via post hoc interpretation techniques are reported to enhance the ability to provide explainable information about sensor-locked activity across multiple BCI systems of diverse nature, introducing techniques like kernel visualizations, saliency map, EEG decoding components, score-weighted maps, ablation tests, among others [29][30][31]. Still, the approaches for building class activation mapping-based (CAM) visualizations have increased interest in MI research, which performs a weighted sum of the feature maps of the last convolutional layer for each class using and a structural regularizer for preventing overfitting during training [32,33].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Image-Based Learning Using Gradient Class Activation Maps for Enhanced Physiological Interpretability of Motor Imagery Skills

Collazos-Huertas

Álvarez-Meza

Castellanos-Domínguez

2022

Applied Sciences

View full text Add to dashboard Cite

Brain activity stimulated by the motor imagery paradigm (MI) is measured by Electroencephalography (EEG), which has several advantages to be implemented with the widely used Brain–Computer Interfaces (BCIs) technology. However, the substantial inter/intra variability of recorded data significantly influences individual skills on the achieved performance. This study explores the ability to distinguish between MI tasks and the interpretability of the brain’s ability to produce elicited mental responses with improved accuracy. We develop a Deep and Wide Convolutional Neuronal Network fed by a set of topoplots extracted from the multichannel EEG data. Further, we perform a visualization technique based on gradient-based class activation maps (namely, GradCam++) at different intervals along the MI paradigm timeline to account for intra-subject variability in neural responses over time. We also cluster the dynamic spatial representation of the extracted maps across the subject set to come to a deeper understanding of MI-BCI coordination skills. According to the results obtained from the evaluated GigaScience Database of motor-evoked potentials, the developed approach enhances the physiological explanation of motor imagery in aspects such as neural synchronization between rhythms, brain lateralization, and the ability to predict the MI onset responses and their evolution during training sessions.

show abstract

Section: Preprocessing and Feature Extraction Of Image-based Represen...mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Image-Based Learning Using Gradient Class Activation Maps for Enhanced Physiological Interpretability of Motor Imagery Skills

Collazos-Huertas

Álvarez-Meza

Castellanos-Domínguez

2022

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…The most commonly method (Saliency) is also the most simplest wherein the gradient w.r.t. input was computed 31 , 32 , 33 , 34 , 35 , 36 , 37 , 34 , 38 . The next commonly used method is plotting the convolutional filters directly; usually, the convolutional filters that have a kernel spanning the entire EEG channels (spatial convolutional layer weights) 39 , 40 , 41 , 42 .…”

Section: Introductionmentioning

confidence: 99%

An Empirical Comparison of Deep Learning Explainability Approaches for EEG using Simulated Ground Truth

Ravindran

Contreras-Vidal

2023

Preprint

View full text Add to dashboard Cite

Recent advancements in machine learning and deep learning (DL) based neural decoders have significantly improved decoding capabilities using scalp electroencephalography (EEG). However, the interpretability of DL models remains an under-explored area. In this study, we compared multiple model explanation methods to identify the most suitable method for EEG and understand when some of these approaches might fail. A simulation framework was developed to evaluate the robustness and sensitivity of twelve back-propagation-based visualization methods by comparing to ground truth features. Multiple methods tested here showed reliability issues after randomizing either model weights or labels: e.g., the saliency approach, which is the most used visualization technique in EEG, was not class or model-specific. We found that DeepLift was consistently accurate as well as robust to detect the three key attributes tested here (temporal, spatial, and spectral precision). Overall, this study provides a review of model explanation methods for DL-based neural decoders and recommendations to understand when some of these methods fail and what they can capture in EEG.

show abstract

“…The correct recognition of neuronal activity in different motion imagination can lead to brain instructions that can help patients with severe motion neuron disease to control external equipment such as wheelchairs. Also, motion imagination classification is also an important support for rehabilitation training [ 5 ].…”

Section: Introductionmentioning

confidence: 99%

“…Li et al proposed an end-to-end spatial-temporal GCNN, which simultaneously captured the spatial-temporal features of EEG signals to identify different motion imagination [13]. Lun et al [14] proposed a deep learning framework based on GCNN by combining the functional topological relationship of electrodes, so as to improve the decoding performance of motion imagination EEG signals. Sun et al proposed an adaptive spatial-temporal GCNN, which can make full use of the characteristics of EEG signal in time domain and channel correlation in space domain [15].…”

Section: Introductionmentioning

confidence: 99%

Considerate motion imagination classification method using deep learning

Yan

Yang²,

Yu³

2022

PLoS ONE

View full text Add to dashboard Cite

In order to improve the classification accuracy of motion imagination, a considerate motion imagination classification method using deep learning is proposed. Specifically, based on a graph structure suitable for electroencephalography as input, the proposed model can accurately represent the distribution of electroencephalography electrodes in non-Euclidean space and fully consider the spatial correlation between electrodes. In addition, the spatial-spectral-temporal multi-dimensional feature information was extracted from the spatial-temporal graph representation and spatial-spectral graph representation transformed from the original electroencephalography signal using the dual branch architecture. Finally, the attention mechanism and global feature aggregation module were designed and combined with graph convolution to adaptively capture the dynamic correlation intensity and effective feature of electroencephalography signals in various dimensions. A series of contrast experiments and ablation experiments on several different public brain-computer interface datasets demonstrated that the excellence of proposed method. It is worth mentioning that, the proposed model is a general framework for the classification of electroencephalography signals, which is suitable for emotion recognition, sleep staging and other fields based on electroencephalography research. Moreover, the model has the potential to be applied in the medical field of motion imagination rehabilitation in real life.

show abstract

A channel-mixing convolutional neural network for motor imagery EEG decoding and feature visualization

Cited by 19 publications

References 36 publications

Image-Based Learning Using Gradient Class Activation Maps for Enhanced Physiological Interpretability of Motor Imagery Skills

Image-Based Learning Using Gradient Class Activation Maps for Enhanced Physiological Interpretability of Motor Imagery Skills

An Empirical Comparison of Deep Learning Explainability Approaches for EEG using Simulated Ground Truth

Considerate motion imagination classification method using deep learning

Contact Info

Product

Resources

About