Aggregating explanation methods for stable and robust explainability

Rieger, Laura; Hansen, Lars Kai

doi:10.48550/arxiv.1903.00519

Cited by 3 publications

(3 citation statements)

References 28 publications

(81 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this work we address the quantification of the surrogate explanation uncertainty by aggregating multiple surrogate coefficients and measuring the consensus among the surrogate explainers. The use of a consensus mechanism to obtain explanations that are less sensitive to sampling variance (further discussed in Section 3) has been proposed in [4,26]. Specifically, [6] and [5] consider aggregating surrogate coefficients in the form of simple ranking schemes inspired from the social sciences and economics.…”

Section: Related Workmentioning

confidence: 99%

Uncertainty Quantification of Surrogate Explanations: an Ordinal Consensus Approach

Schulz¹,

Poyiadzi²,

Santos-Rodríguez³

2022

nldl

View full text Add to dashboard Cite

Explainability of black-box machine learning models is crucial, in particular when deployed in critical applications such as medicine or autonomous cars. Existing approaches produce explanations for the predictions of models, however, how to assess the quality and reliability of such explanations remains an open question. In this paper we take a step further in order to provide the practitioner with tools to judge the trustworthiness of an explanation. To this end, we produce estimates of the uncertainty of a given explanation by measuring the ordinal consensus amongst a set of diverse bootstrapped surrogate explainers. While we encourage diversity by using ensemble techniques, we propose and analyse metrics to aggregate the information contained within the set of explainers through a rating scheme. We empirically illustrate the properties of this approach through experiments on state-of-the-art Convolutional Neural Network ensembles. Furthermore, through tailored visualisations, we show specific examples of situations where uncertainty estimates offer concrete actionable insights to the user beyond those arising from standard surrogate explainers.

show abstract

Section: Related Workmentioning

confidence: 99%

Uncertainty Quantification of Surrogate Explanations: an Ordinal Consensus Approach

Schulz¹,

Poyiadzi²,

Santos-Rodríguez³

2022

nldl

View full text Add to dashboard Cite

show abstract

“…Lakkaraju and Bastani (2020) conducted a thought-provoking study on misleading effects of manipulated model explanations which provide arguments for why such research becomes crucial to achieve responsibility in machine learning use. Rieger and Hansen (2019) present a defence strategy against the attack via data change of Dombrowski et al (2019). The main idea is to aggregate various model explanations, which produces robust results without changing the model.…”

Section: Related Workmentioning

confidence: 99%

Fooling Partial Dependence via Data Poisoning

Baniecki¹,

Kretowicz²,

Biecek³

2021

Preprint

View full text Add to dashboard Cite

Many methods have been developed to understand complex predictive models and high expectations are placed on post-hoc model explainability. It turns out that such explanations are not robust nor trustworthy, and they can be fooled. This paper presents techniques for attacking Partial Dependence (plots, profiles, PDP), which are among the most popular methods of explaining any predictive model trained on tabular data. We showcase that PD can be manipulated in an adversarial manner, which is alarming, especially in financial or medical applications where auditability became a must-have trait supporting black-box models. The fooling is performed via poisoning the data to bend and shift explanations in the desired direction using genetic and gradient algorithms. To the best of our knowledge, this is the first work performing attacks on variable dependence explanations. The novel approach of using a genetic algorithm for doing so is highly transferable as it generalizes both ways: in a model-agnostic and an explanation-agnostic manner.

show abstract

“…In this work we address the quantification of the surrogate explanation uncertainty by aggregating multiple surrogate coefficients. The use of a consensus mechanism to obtain explanations that are less sensitive to sampling variance (further discussed in Section 3) has been proposed in [5,24]. Specifically, [6] and [4] consider aggregating surrogate coefficients in the form of simple ranking schemes inspired from the social sciences and economics.…”

Section: Related Workmentioning

confidence: 99%

Uncertainty Quantification of Surrogate Explanations: an Ordinal Consensus Approach

Schulz¹,

Poyiadzi²,

Santos-Rodríguez³

2021

Preprint

View full text Add to dashboard Cite

Explainability of black-box machine learning models is crucial, in particular when deployed in critical applications such as medicine or autonomous cars. Existing approaches produce explanations for the predictions of models, however, how to assess the quality and reliability of such explanations remains an open question. In this paper we take a step further in order to provide the practitioner with tools to judge the trustworthiness of an explanation. To this end, we produce estimates of the uncertainty of a given explanation by measuring the ordinal consensus amongst a set of diverse bootstrapped surrogate explainers. While we encourage diversity by using ensemble techniques, we propose and analyse metrics to aggregate the information contained within the set of explainers through a rating scheme. We empirically illustrate the properties of this approach through experiments on state-of-theart Convolutional Neural Network ensembles. Furthermore, through tailored visualisations, we show specific examples of situations where uncertainty estimates offer concrete actionable insights to the user beyond those arising from standard surrogate explainers.

show abstract

Aggregating explanation methods for stable and robust explainability

Cited by 3 publications

References 28 publications

Uncertainty Quantification of Surrogate Explanations: an Ordinal Consensus Approach

Uncertainty Quantification of Surrogate Explanations: an Ordinal Consensus Approach

Fooling Partial Dependence via Data Poisoning

Uncertainty Quantification of Surrogate Explanations: an Ordinal Consensus Approach

Contact Info

Product

Resources

About