Methods for interpreting and understanding deep neural networks

Montavon, Grégoire; Samek, Wojciech; Müller, Klaus Robert

doi:10.1016/j.dsp.2017.10.011

Cited by 2,078 publications

(1,457 citation statements)

References 42 publications

Supporting

Mentioning

1,435

Contrasting

Unclassified

Order By: Relevance

“…Activation maximization (Montavon et al, ) identifies input patterns that lead to maximal activations relating to specific classes in the output layer (Berkes & Wiskott, ; Simonyan & Zisserman, ). This makes the visualization of prototypes of classes possible, and assesses which properties the model captures for classes (Erhan, Bengio, Courville, & Vincent, ).…”

Section: General Approaches Of Explainable Ai Modelsmentioning

confidence: 99%

See 1 more Smart Citation

Causability and explainability of artificial intelligence in medicine

Holzinger

Langs

Denk

et al. 2019

WIREs Data Min & Knowl

951

571

View full text Add to dashboard Cite

Explainable artificial intelligence (AI) is attracting much interest in medicine. Technically, the problem of explainability is as old as AI itself and classic AI represented comprehensible retraceable approaches. However, their weakness was in dealing with uncertainties of the real world. Through the introduction of probabilistic learning, applications became increasingly successful, but increasingly opaque. Explainable AI deals with the implementation of transparency and traceability of statistical black‐box machine learning methods, particularly deep learning (DL). We argue that there is a need to go beyond explainable AI. To reach a level of explainable medicine we need causability. In the same way that usability encompasses measurements for the quality of use, causability encompasses measurements for the quality of explanations. In this article, we provide some necessary definitions to discriminate between explainability and causability as well as a use‐case of DL interpretation and of human explanation in histopathology. The main contribution of this article is the notion of causability, which is differentiated from explainability in that causability is a property of a person, while explainability is a property of a system This article is categorized under: Fundamental Concepts of Data and Knowledge > Human Centricity and User Interaction

show abstract

Section: General Approaches Of Explainable Ai Modelsmentioning

confidence: 99%

“…The effect of the ℓ 2 ‐norm regularizer in the code space can instead be understood as encouraging codes that have high probability. High probability codes do not necessarily map to high density regions of the input space; for more details refer to the excellent tutorial given by Montavon et al ().…”

Section: General Approaches Of Explainable Ai Modelsmentioning

confidence: 99%

Causability and explainability of artificial intelligence in medicine

Holzinger

Langs

Denk

et al. 2019

WIREs Data Min & Knowl

951

571

View full text Add to dashboard Cite

show abstract

“…To understand class-specific spectral characteristics in the EEG recordings, we analyzed band powers in five frequency ranges: delta (0-4 Hz), theta (4-8 Hz), alpha (8)(9)(10)(11)(12)(13)(14), low beta (14)(15)(16)(17)(18)(19)(20), high beta (20)(21)(22)(23)(24)(25)(26)(27)(28)(29)(30) and low gamma .…”

Section: Visualizations Of the Spectral Differences Between Normalmentioning

confidence: 99%

“…Deep learning models that use an attention mechanism might be more interpretable, since these models can highlight which parts of the recording were most important for the decoding decision. Other deep learning visualization methods like recent saliency map methods [27,28] to explain individual decisions or conditional generative adversarial networks [29,30] to understand what makes a recording pathological or normal might further improve the clinical benefit of deep learning methods that decode pathological EEG.…”

mentioning

confidence: 99%

Deep learning with convolutional neural networks for decoding and visualization of EEG pathology

Schirrmeister

Gemein

Eggensperger

et al. 2017

2017 IEEE Signal Processing in Medicine and Biology Symposium (SPMB)

145

View full text Add to dashboard Cite

Abstract-We apply convolutional neural networks (ConvNets) to the task of distinguishing pathological from normal EEG recordings in the Temple University Hospital EEG Abnormal Corpus. We use two basic, shallow and deep ConvNet architectures recently shown to decode task-related information from EEG at least as well as established algorithms designed for this purpose. In decoding EEG pathology, both ConvNets reached substantially better accuracies (about 6% better, ≈85% vs. ≈79%) than the only published result for this dataset, and were still better when using only 1 minute of each recording for training and only six seconds of each recording for testing. We used automated methods to optimize architectural hyperparameters and found intriguingly different ConvNet architectures, e.g., with max pooling as the only nonlinearity. Visualizations of the ConvNet decoding behavior showed that they used spectral power changes in the delta (0-4 Hz) and theta (4-8 Hz) frequency range, possibly alongside other features, consistent with expectations derived from spectral analysis of the EEG data and from the textual medical reports. Analysis of the textual medical reports also highlighted the potential for accuracy increases by integrating contextual information, such as the age of subjects. In summary, the ConvNets and visualization techniques used in this study constitute a next step towards clinically useful automated EEG diagnosis and establish a new baseline for future work on this topic.I. I Electroencephalography (EEG) is widely used in clinical practice because of its low cost and its lack of side effects due to its noninvasive nature. It is important both as a screening method as well as for hypothesis-based diagnostics, e.g., in epilepsy or stroke. One of the main limitations of using EEG for diagnostics is the required time and specialized knowledge of experts that need to be well-trained on EEG diagnostics to reach reliable results. Therefore, a machine-learning approach that aids in the diagnostic process could make EEG diagnosis more widely accessible, reduce time and effort for clinicians and potentially make diagnoses more accurate.In recent years researchers have increasingly addressed the field of computer-aided EEG diagnosis. So far, the applications were mostly limited to specific diagnoses such as Alzheimer's disease regression, neural networks, and more. This large variety of used methods indicates that the search for the best decoding approach for diverse types of EEG diagnosis is still ongoing.To overcome the lack of large datasets representative of the variety of EEG-diagnosable diseases and the heterogeneity of clinical populations, the Temple University Hospital (TUH) has published an unprecedented public dataset of clinical EEG recordings [6]. From this dataset with over 16000 clinical recordings, the TUH Abnormal EEG Corpus with about 3000 recordings has been created specifically to foster the development of methods for distinguishing pathological from normal EEG. Due to its size and rich annotatio...

show abstract

“…Another future direction would be to analyze the interpretability of NNS systems, specifically for recommender systems with non-linear query mechanism, in terms of salient features that have led to the query result. This is in the line of research on ''explaining learning machines'', i.e., answering to the question which part of the data is responsible for specific decisions made by learning machines (Baehrens et al 2010;Zeiler and Fergus 2014;Bach et al 2015; Ribeiro et al 2016; Montavon et al 2017Montavon et al , 2018. This question is non-trivial when the learning machines are complex and non-linear.…”

Section: Resultsmentioning

confidence: 99%

Sharing hash codes for multiple purposes

Pronobis

Panknin

Kirschnick

et al. 2018

Jpn J Stat Data Sci

Self Cite

View full text Add to dashboard Cite

Locality sensitive hashing (LSH) is a powerful tool in data science, which enables sublinear-time approximate nearest neighbor search. A variety of hashing schemes have been proposed for different dissimilarity measures. However, hash codes significantly depend on the dissimilarity, which prohibits users from adjusting the dissimilarity at query time. In this paper, we propose multiple purpose LSH (mp-LSH) which shares the hash codes for different dissimilarities. mp-LSH supports L2, cosine, and inner product dissimilarities, and their corresponding weighted sums, where the weights can be adjusted at query time. It also allows us to modify the importance of pre-defined groups of features. Thus, mp-LSH enables us, for example, to retrieve similar items to a query with the user preference taken into account, to find a similar material to a query with some properties (stability, utility, etc.) optimized, and to turn on or off a part of multi-modal information (brightness, color, audio, text, etc.) in image/video retrieval. We theoretically and empirically analyze the performance of three variants of mp-LSH, and demonstrate their usefulness on real-world data sets.

show abstract

Methods for interpreting and understanding deep neural networks

Cited by 2,078 publications

References 42 publications

Causability and explainability of artificial intelligence in medicine

Causability and explainability of artificial intelligence in medicine

Deep learning with convolutional neural networks for decoding and visualization of EEG pathology

Sharing hash codes for multiple purposes

Contact Info

Product

Resources

About