ERIC: Extracting Relations Inferred from Convolutions

Townsend, J.E.; Kasioumis, Theodoros; Inakoshi, Hiroya

doi:10.1007/978-3-030-69535-4_13

Cited by 7 publications

(11 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These literals were then used to generate symbolic rules by means of a logic program that approximates the behavior of the convolutional layer(s) with respect to the CNN’s output. The approximation M * of the original CNN was reported to achieve high classification accuracy and fidelity in [ 12 ]. The results were also evaluated in terms of the sizes of the extracted rule sets, with smaller sets considered to be more human-comprehensible.…”

Section: Related Workmentioning

confidence: 99%

“…This work used the ERIC method [ 12 , 39 ] for extracting rules from CNN kernels. Based on the ERIC framework, the last convolutional layer, l , of the trained VGG16, M , was quantized and binarized to produce literals.…”

Section: Interactive Model Explanation and Interventionmentioning

confidence: 99%

“…In this work, we build on the previous research to extract rules from trained CNNs and investigate the semantically-meaningful relationships among the concepts referred to by these rules. The ERIC framework (Extracting Relations Inferred from Convolutions) [ 12 ] is used as the rule extraction method, as described below.…”

Section: Introductionmentioning

confidence: 99%

“…ERIC has been shown empirically to generate rules that can approximate the original CNN, as measured by high fidelity scores, i.e. the accuracy of the rules relative to the CNN rather than the data labels [ 12 ]. Each rule takes the form L 1 ∧ L 2 ∧ … ∧ L n → A , denoting a conjunction of literals L i , 1 ≤ i ≤ n (each of which is either a propositional atom or its negation) that imply an atom A (if L i hold true then A holds true).…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

From explanation to intervention: Interactive knowledge extraction from Convolutional Neural Networks used in radiology

Ngan,

Mansouri-Benssassi,

Phelan

et al. 2024

PLoS ONE

View full text Add to dashboard Cite

Deep Learning models such as Convolutional Neural Networks (CNNs) are very effective at extracting complex image features from medical X-rays. However, the limited interpretability of CNNs has hampered their deployment in medical settings as they failed to gain trust among clinicians. In this work, we propose an interactive framework to allow clinicians to ask what-if questions and intervene in the decisions of a CNN, with the aim of increasing trust in the system. The framework translates a layer of a trained CNN into a measurable and compact set of symbolic rules. Expert interactions with visualizations of the rules promote the use of clinically-relevant CNN kernels and attach meaning to the rules. The definition and relevance of the kernels are supported by radiomics analyses and permutation evaluations, respectively. CNN kernels that do not have a clinically-meaningful interpretation are removed without affecting model performance. By allowing clinicians to evaluate the impact of adding or removing kernels from the rule set, our approach produces an interpretable refinement of the data-driven CNN in alignment with medical best practice.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Interactive Model Explanation and Interventionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

From explanation to intervention: Interactive knowledge extraction from Convolutional Neural Networks used in radiology

Ngan,

Mansouri-Benssassi,

Phelan

et al. 2024

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…Explainability in neuro-symbolic systems has been traditionally approached by learning a set of symbolic rules, known as Knowledge Extraction, and evaluating how well it may approximate the behaviour of a complex neural network by measuring the percentage of matching predictions on a test set, referred to as fidelity [41,35] This is comparable to most contemporary explainability methods that are not powerful enough to guarantee the soundness and completeness of the explanation concerning the underlying model. Most metrics currently in place are lacking a reliable way of expressing this uncertainty.…”

Section: Xai In Neural-symbolic Aimentioning

confidence: 99%

A Practical guide on Explainable AI Techniques applied on Biomedical use case applications

Bennetot¹,

Donadello²,

Qadi³

et al. 2021

Preprint

View full text Add to dashboard Cite

Last years have been characterized by an upsurge of opaque automatic decision support systems, such as Deep Neural Networks (DNNs). Although they have great generalization and prediction skills, their functioning does not allow obtaining detailed explanations of their behaviour. As opaque machine learning models are increasingly being employed to make important predictions in critical environments, the danger is to create and use decisions that are not justifiable or legitimate. Therefore, there is a general agreement on the importance of endowing machine learning models with explainability. The reason is that EXplainable Artificial Intelligence (XAI) techniques can serve to verify and certify model outputs and enhance them with desirable notions such as trustworthiness, accountability, transparency and fairness. This tutorial is meant to be the go-to handbook for any audience with a computer science background aiming at getting intuitive insights of machine learning models, accompanied with straight, fast, and intuitive explanations out of the box. We believe that this article provide a valuable contribution for applying XAI techniques in their particular day-to-day models, datasets and use-cases. Figure 1 acts as a flowchart/map for the reader and should help him to find the ideal method to use according to his type of data. The reader will find a description of the proposed method as well as an example of use and a Python notebook that he can easily modify as he pleases in order to apply it to his own case of application.

show abstract

Using Logic Programming and Kernel-Grouping for Improving Interpretability of Convolutional Neural Networks

Padalkar,

Wang,

Gupta

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

ERIC: Extracting Relations Inferred from Convolutions

Cited by 7 publications

References 23 publications

From explanation to intervention: Interactive knowledge extraction from Convolutional Neural Networks used in radiology

From explanation to intervention: Interactive knowledge extraction from Convolutional Neural Networks used in radiology

A Practical guide on Explainable AI Techniques applied on Biomedical use case applications

Using Logic Programming and Kernel-Grouping for Improving Interpretability of Convolutional Neural Networks

Contact Info

Product

Resources

About