Model Explainability in Deep Learning Based Natural Language Processing

Gholizadeh, Shafie; Zhou, Nengfeng

doi:10.48550/arxiv.2106.07410

Cited by 4 publications

(4 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where indicates a concatenation operator and dim(s r ) = R × R × R. Note that the self-connections corresponding to the seed-ROI are excluded during the calculation of the relevance score. To estimate the local contributions of the r-th ROI, we simply aggregate the relevance scores for various connections [42]. This can be done using the following equation…”

Section: Connection-wise Relevance Score Estimationmentioning

confidence: 99%

“…We perform statistical analysis to distinguish individual impacts and identify the most important effects [42]. Therefore, given the averaged relevance scores S ∈ R R×R , we reformulate the ROI-level representative vectors (i.e., f v ∈ R R×1 and f c ∈ R R×1 ) using statistical measures such as mean and count, as referenced in Algorithm 2.…”

Section: Roi Selection Network and Diagnostic Classifier 1) Roi Selec...mentioning

confidence: 99%

See 1 more Smart Citation

EAG-RS: A Novel Explainability-Guided ROI-Selection Framework for ASD Diagnosis via Inter-Regional Relation Learning

Jung,

Jeon,

Kang

et al. 2024

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

Deep learning models based on resting-state functional magnetic resonance imaging (rs-fMRI) have been widely used to diagnose brain diseases, particularly autism spectrum disorder (ASD). Existing studies have leveraged the functional connectivity (FC) of rs-fMRI, achieving notable classification performance. However, they have significant limitations, including the lack of adequate information while using linear low-order FC as inputs to the model, not considering individual characteristics (i.e., different symptoms or varying stages of severity) among patients with ASD, and the non-explainability of the decision process. To cover these limitations, we propose a novel explainability-guided region of interest (ROI) selection (EAG-RS) framework that identifies non-linear high-order functional associations among brain regions by leveraging an explainable artificial intelligence technique and selects class-discriminative regions for brain disease identification. The proposed framework includes three steps: (i) inter-regional relation learning to estimate nonlinear relations through random seed-based network masking, (ii) explainable connection-wise relevance score estimation to explore high-order relations between functional connections, and (iii) non-linear high-order FC-based diagnosis-informative ROI selection and classifier learning to identify ASD. We validated the effectiveness of our proposed method by conducting experiments using the Autism Brain Imaging Database Exchange (ABIDE) dataset, demonstrating that the proposed method outperforms other comparative methods in terms of various evaluation metrics. Furthermore, we qualitatively analyzed the selected ROIs and identified ASD subtypes linked to previous neuroscientific studies.

show abstract

Section: Connection-wise Relevance Score Estimationmentioning

confidence: 99%

Section: Roi Selection Network and Diagnostic Classifier 1) Roi Selec...mentioning

confidence: 99%

EAG-RS: A Novel Explainability-Guided ROI-Selection Framework for ASD Diagnosis via Inter-Regional Relation Learning

Jung,

Jeon,

Kang

et al. 2024

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

show abstract

“…In this study, a bibliographic search was conducted for publications that employed ML techniques (excluding DL) for the classification of binary data (interictal and ictal) in humans with epilepsy. It was chosen to use ML techniques because they are more feasible for performing model explainability analysis and observing the importance of features [ 60 ]. DL models, being “black-box”, make it difficult to understand predictions and are therefore less interpretable [ 61 ].…”

Section: Related Workmentioning

confidence: 99%

Using Explainable Artificial Intelligence to Obtain Efficient Seizure-Detection Models Based on Electroencephalography Signals

Vieira,

Guedes,

Santos

et al. 2023

Sensors

View full text Add to dashboard Cite

Epilepsy is a condition that affects 50 million individuals globally, significantly impacting their quality of life. Epileptic seizures, a transient occurrence, are characterized by a spectrum of manifestations, including alterations in motor function and consciousness. These events impose restrictions on the daily lives of those affected, frequently resulting in social isolation and psychological distress. In response, numerous efforts have been directed towards the detection and prevention of epileptic seizures through EEG signal analysis, employing machine learning and deep learning methodologies. This study presents a methodology that reduces the number of features and channels required by simpler classifiers, leveraging Explainable Artificial Intelligence (XAI) for the detection of epileptic seizures. The proposed approach achieves performance metrics exceeding 95% in accuracy, precision, recall, and F1-score by utilizing merely six features and five channels in a temporal domain analysis, with a time window of 1 s. The model demonstrates robust generalization across the patient cohort included in the database, suggesting that feature reduction in simpler models—without resorting to deep learning—is adequate for seizure detection. The research underscores the potential for substantial reductions in the number of attributes and channels, advocating for the training of models with strategically selected electrodes, and thereby supporting the development of effective mobile applications for epileptic seizure detection.

show abstract

“…Figure 6: Diagram of the perturbation based method (left, the figure is brought from[61]) and LRP (right, the figure is brought from[64]). Left: At each step, the least important word is removed by the saliency score using the leave-one-out technique.…”

mentioning

confidence: 99%

Interpreting Deep Learning Models in Natural Language Processing: A Review

Sun¹,

Yang²,

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

Neural network models have achieved state-of-the-art performances in a wide range of natural language processing (NLP) tasks. However, a long-standing criticism against neural network models is the lack of interpretability, which not only reduces the reliability of neural NLP systems but also limits the scope of their applications in areas where interpretability is essential (e.g., health care applications). In response, the increasing interest in interpreting neural NLP models has spurred a diverse array of interpretation methods over recent years. In this survey, we provide a comprehensive review of various interpretation methods for neural models in NLP. We first stretch out a high-level taxonomy for interpretation methods in NLP, i.e., training-based approaches, test-based approaches and hybrid approaches. Next, we describe sub-categories in each category in detail, e.g., influence-function based methods, KNN-based methods, attention-based models, saliency-based methods, perturbation-based methods, etc. We point out deficiencies of current methods and suggest some avenues for future research.

show abstract

Model Explainability in Deep Learning Based Natural Language Processing

Cited by 4 publications

References 17 publications

EAG-RS: A Novel Explainability-Guided ROI-Selection Framework for ASD Diagnosis via Inter-Regional Relation Learning

EAG-RS: A Novel Explainability-Guided ROI-Selection Framework for ASD Diagnosis via Inter-Regional Relation Learning

Using Explainable Artificial Intelligence to Obtain Efficient Seizure-Detection Models Based on Electroencephalography Signals

Interpreting Deep Learning Models in Natural Language Processing: A Review

Contact Info

Product

Resources

About