Interpretable SincNet-based Deep Learning for Emotion Recognition from EEG brain activity

Mayor-Torres, Juan Manuel; Ravanelli, Mirco; Medina-DeVilliers, Sara; Lerner, Matthew D.; Riccardi, Giuseppe

doi:10.1109/embc46164.2021.9630427

Cited by 15 publications

(7 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The EEG signal was recorded in adherence with best EEG data-collection guidelines [36], [37], [38] during overground walking using a wireless 32-channel gtec g.Nautilus active device, with a sampling rate of 250Hz. The EEG channels included were FP1, FP2, AF3, AF4, F7, F3, Fz, F4, F8, FC5, FC1, FC2, FC6, T7, C3, Cz, C4, T8, CP5, CP1, CP2, CP6, P7, P3, Pz, P4, P8, PO7, PO3, PO4, PO8, and Oz.…”

Section: Methodsmentioning

confidence: 99%

Robotic-Assisted Gait for lower-limb Rehabilitation: Evidence of Altered Neural Mechanisms in Stroke

Mayor-Torres

O’Callaghan

Korik

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Robotic-Assisted Gait training (RAGT) offers an innovative therapeutic option for restoration of functional gait in stroke survivors, complementing existing physical rehabilitation strategies. However, there is a limited understanding of the neurophysiological response induced by this training in end-users. Neural desynchronization and Cortico-Muscular Coherence (CMC) are two biomarkers that define the level of muscle-cortex association during gait phases and can be used to estimate induced user's adaptation during RAGT. In this study, we measure Event-Related Spectral Perturbation (ERSP) and CMC from three healthy individuals and three stroke survivors during overground-gait with and without an exoskeleton. Results show that (1) the use of the exoskeleton in healthy individuals is associated with a different and more refined motor-control represented in a high θ-desynchronization, (2) altered and noisy ERSP and lower and non-focal β-CMC patterns are observed in Stroke patients when performing overground-gait both with and without the Exoskeleton, and (3) Exoskeleton use in stroke survivors is associated with a reduction in swing-time during gait-cycle, but this effect is not correlated with an increment of θ-desynchronization and/or β-CMC. ERSP and CMC demonstrated evidence of neural modulation in able-bodied users during RAGT, which could not be detected in subacute stroke survivors during RAGT. These results suggest that the gait-parameters changes observed during exoskeleton use in subacute stroke survivors are unlikely to be neurally driven.

show abstract

Section: Methodsmentioning

confidence: 99%

Robotic-Assisted Gait for lower-limb Rehabilitation: Evidence of Altered Neural Mechanisms in Stroke

Mayor-Torres

O’Callaghan

Korik

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…For explainability, we used the αβ-rule (70) of layer-wise relevance propagation (LRP) (71,72). LRP is a popular approach that have been used in many studies for insight into neurological time-series and neuroimaging data (8,9,59,60,(73)(74)(75)(76)(77)(78)(79)(80). LRP involves several steps.…”

Section: Description Of Explainability Approachmentioning

confidence: 99%

Pairing Explainable Deep Learning Classification with Clustering to Uncover Effects of Schizophrenia Upon Whole Brain Functional Network Connectivity Dynamics

Ellis

Miller

Calhoun

2023

Preprint

View full text Add to dashboard Cite

Many studies have analyzed resting state functional magnetic resonance imaging (rs-fMRI) dynamic functional network connectivity (dFNC) data to elucidate the effects of neurological and neuropsychiatric disorders upon the interactions of brain regions over time. Existing studies often use either machine learning classification or clustering algorithms. Additionally, several studies have used clustering algorithms to extract features related to brain states trajectories that can be used to train interpretable classifiers. However, the combination of explainable dFNC classifiers followed by clustering algorithms is highly underutilized. In this study, we show how such an approach can be used to study the effects of schizophrenia (SZ) upon brain activity. Specifically, we train an explainable deep learning model to classify between individuals with SZ and healthy controls. We then cluster the resulting explanations, identifying discriminatory states of dFNC. We lastly apply several novel measures to quantify aspects of the classifier explanations and obtain additional insights into the effects of SZ upon brain network dynamics. Specifically, we uncover effects of schizophrenia upon subcortical, sensory, and cerebellar network interactions. We also find that individuals with SZ likely have reduced variability in overall brain activity and that the effects of SZ may be temporally localized. In addition to uncovering effects of SZ upon brain network dynamics, our approach could provide novel insights into a variety of neurological and neuropsychiatric disorders in future dFNC studies.

show abstract

“…Liu et al [22] propose Gated Bi-directional Alignment Network that effectively captures speech-text relations, and an interpretable Group Gated Fusion (GGF) layer that determines the significance of each modality through contribution weights. Mayou et al [23] propose a SincNet-based network for emotion classification with EEG signals that is interpreted by inspecting the filters learned by the model. Nguyen et al [24] introduce a novel DNN architecture for multimodal emotion recognition and use nonlinear Gaussian Additive Models to interpret the same.…”

Section: B Emotion Aimentioning

confidence: 99%

Interpretability for Multimodal Emotion Recognition using Concept Activation Vectors

Asokan¹,

Kumar²,

Ragam³

et al. 2022

Preprint

View full text Add to dashboard Cite

Multimodal Emotion Recognition refers to the classification of input video sequences into emotion labels based on multiple input modalities (usually video, audio and text). In recent years, Deep Neural networks have shown remarkable performance in recognizing human emotions, and are on par with human-level performance on this task. Despite the recent advancements in this field, emotion recognition systems are yet to be accepted for real world setups due to the obscure nature of their reasoning and decision-making process. Most of the research in this field deals with novel architectures to improve the performance for this task, with a few attempts at providing explanations for these models' decisions. In this paper, we address the issue of interpretability for neural networks in the context of emotion recognition using Concept Activation Vectors (CAVs). To analyse the model's latent space, we define humanunderstandable concepts specific to Emotion AI and map them to the widely-used IEMOCAP multimodal database. We then evaluate the influence of our proposed concepts at multiple layers of the Bi-directional Contextual LSTM (BC-LSTM) network to show that the reasoning process of neural networks for emotion recognition can be represented using human-understandable concepts. Finally, we perform hypothesis testing on our proposed concepts to show that they are significant for interpretability of this task.

show abstract

Interpretable SincNet-based Deep Learning for Emotion Recognition from EEG brain activity

Cited by 15 publications

References 15 publications

Robotic-Assisted Gait for lower-limb Rehabilitation: Evidence of Altered Neural Mechanisms in Stroke

Robotic-Assisted Gait for lower-limb Rehabilitation: Evidence of Altered Neural Mechanisms in Stroke

Pairing Explainable Deep Learning Classification with Clustering to Uncover Effects of Schizophrenia Upon Whole Brain Functional Network Connectivity Dynamics

Interpretability for Multimodal Emotion Recognition using Concept Activation Vectors

Contact Info

Product

Resources

About