Explanation Methods in Deep Learning: Users, Values, Concerns and Challenges

Ras, Gabriëlle; Gerven, Marcel van; Haselager, Pim

doi:10.1007/978-3-319-98131-4_2

Cited by 156 publications

(115 citation statements)

References 52 publications

Supporting

Mentioning

107

Contrasting

Order By: Relevance

“…The use of big data to train AI-models has, to name a few, been adopted by government agencies, institutions of law, human resources, and medical decision systems. A number of such models have been shown to make decisions based on the gender or ancestral origin of an individual, leading to concerns about their "fairness" [15,16,17]. With the recent enforcement of the General Data Protection Regulation laws 4 , individuals have the right to know the rationale behind an automated decision concerning them.…”

Section: Introductionmentioning

confidence: 99%

Turning a Blind Eye: Explicit Removal of Biases and Variation from Deep Neural Network Embeddings

Alvi

Zisserman

Nellåker

2019

Lecture Notes in Computer Science

180

217

View full text Add to dashboard Cite

Neural networks achieve the state-of-the-art in image classification tasks. However, they can encode spurious variations or biases that may be present in the training data. For example, training an age predictor on a dataset that is not balanced for gender can lead to gender biased predicitons (e.g. wrongly predicting that males are older if only elderly males are in the training set). We present two distinct contributions: 1) An algorithm that can remove multiple sources of variation from the feature representation of a network. We demonstrate that this algorithm can be used to remove biases from the feature representation, and thereby improve classification accuracies, when training networks on extremely biased datasets. 2) An ancestral origin database of 14,000 images of individuals from East Asia, the Indian subcontinent, sub-Saharan Africa, and Western Europe. We demonstrate on this dataset, for a number of facial attribute classification tasks, that we are able to remove racial biases from the network feature representation.

show abstract

Section: Introductionmentioning

confidence: 99%

Turning a Blind Eye: Explicit Removal of Biases and Variation from Deep Neural Network Embeddings

Alvi

Zisserman

Nellåker

2019

Lecture Notes in Computer Science

180

217

View full text Add to dashboard Cite

show abstract

“…Indeed, after network training it is very difficult to get back into the feature-set used to define the model. For these reasons, this study was based on machine learning algorithms, which are more transparent and explainable compared to deep learning ones [17].…”

Section: Introductionmentioning

confidence: 99%

Double-Step Machine Learning Based Procedure for HFOs Detection and Classification

Sciaraffa

Klados²,

Borghini

et al. 2020

Brain Sciences

View full text Add to dashboard Cite

The need for automatic detection and classification of high-frequency oscillations (HFOs) as biomarkers of the epileptogenic tissue is strongly felt in the clinical field. In this context, the employment of artificial intelligence methods could be the missing piece to achieve this goal. This work proposed a double-step procedure based on machine learning algorithms and tested it on an intracranial electroencephalogram (iEEG) dataset available online. The first step aimed to define the optimal length for signal segmentation, allowing for an optimal discrimination of segments with HFO relative to those without. In this case, binary classifiers have been tested on a set of energy features. The second step aimed to classify these segments into ripples, fast ripples and fast ripples occurring during ripples. Results suggest that LDA applied to 10 ms segmentation could provide the highest sensitivity (0.874) and 0.776 specificity for the discrimination of HFOs from no-HFO segments. Regarding the three-class classification, non-linear methods provided the highest values (around 90%) in terms of specificity and sensitivity, significantly different to the other three employed algorithms. Therefore, this machine-learning-based procedure could help clinicians to automatically reduce the quantity of irrelevant data.

show abstract

“…(explanation, justification, transparency, etc.) Several recent papers have tried to clarify those expressions [20,3,21,4,22,2,23,5] and separate the various approaches in two goals: build interpretable models and/or provide justification of the prediction. [24] for instance, described an interpretable proxy (a decision tree) able to explain the logic of each prediction of a pretrained convolutional neural networks.…”

Section: Related Workmentioning

confidence: 99%

“…Reliably evaluating the quality of an explanation is not straightforward [3,4,2,5]. In this work, we propose to evaluate the explainability power of the semantic bottleneck by measuring its capacity to detect failure of the prediction function, either through an automated detector as [6], or through human judgment.…”

Section: Introductionmentioning

confidence: 99%

Semantic Bottleneck for Computer Vision Tasks

Bucher

Herbin

Jurie

2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

This paper introduces a novel method for the representation of images that is semantic by nature, addressing the question of computation intelligibility in computer vision tasks. More specifically, our proposition is to introduce what we call a semantic bottleneck in the processing pipeline, which is a crossing point in which the representation of the image is entirely expressed with natural language, while retaining the efficiency of numerical representations. We show that our approach is able to generate semantic representations that give state-of-theart results on semantic content-based image retrieval and also perform very well on image classification tasks. Intelligibility is evaluated through user centered experiments for failure detection.

show abstract

Explanation Methods in Deep Learning: Users, Values, Concerns and Challenges

Abstract: Issues regarding explainable AI

Cited by 156 publications

References 52 publications

Turning a Blind Eye: Explicit Removal of Biases and Variation from Deep Neural Network Embeddings

Turning a Blind Eye: Explicit Removal of Biases and Variation from Deep Neural Network Embeddings

Double-Step Machine Learning Based Procedure for HFOs Detection and Classification

Semantic Bottleneck for Computer Vision Tasks

Contact Info

Product

Resources

About