Model Interpretability through the Lens of Computational Complexity

Barceló, Pablo; Monet, Mikaël; Pérez, Jorge; Subercaseaux, Bernardo

doi:10.48550/arxiv.2010.12265

“…The importance of evaluating explanation methods has been discussed in the literature [21,50]. There are various attempts to measure different aspects of an explanation: usefulness to humans [18,25,33]; complexity [32]; difficulty of answering queries [7]; and robustness [3]. In this paper, we measure faithfulness to the model.…”

Section: Related Workmentioning

confidence: 99%

Framework for Evaluating Faithfulness of Local Explanations

Dasgupta¹,

Frost²,

Moshkovitz³

2022

Preprint

View full text Add to dashboard Cite

We study the faithfulness of an explanation system to the underlying prediction model. We show that this can be captured by two properties, consistency and sufficiency, and introduce quantitative measures of the extent to which these hold. Interestingly, these measures depend on the test-time data distribution. For a variety of existing explanation systems, such as anchors, we analytically study these quantities. We also provide estimators and sample complexity bounds for empirically determining the faithfulness of black-box explanation systems. Finally, we experimentally validate the new properties and estimators.

show abstract

“…The idea of using complexity as a proxy for interpretability was also proposed in [3], where the authors stated that the computational complexity of a model can be used as a metric of interpretability as it directly resembles the number of operations that must be interpreted by humans.…”

Section: Related Workmentioning

confidence: 99%

Interpretable pipelines with evolutionary optimized modules for reinforcement learning tasks with visual inputs

Custode

¹

,

Iacca

²

2022

Proceedings of the Genetic and Evolutionary Computation Conference Companion

View full text Add to dashboard Cite

The importance of explainability in AI has become a pressing concern, for which several explainable AI (XAI) approaches have been recently proposed. However, most of the available XAI techniques are post-hoc methods, which however may be only partially reliable, as they do not reflect exactly the state of the original models. Thus, a more direct way for achieving XAI is through interpretable (also called glass-box) models. These models have been shown to obtain comparable (and, in some cases, better) performance with respect to black-boxes models in various tasks such as classification and reinforcement learning. However, they struggle when working with raw data, especially when the input dimensionality increases and the raw inputs alone do not give valuable insights on the decision-making process. Here, we propose to use end-to-end pipelines composed of multiple interpretable models co-optimized by means of evolutionary algorithms, that allows us to decompose the decision-making process into two parts: computing high-level features from raw data, and reasoning on the extracted high-level features. We test our approach in reinforcement learning environments from the Atari benchmark, where we obtain comparable results (with respect to black-box approaches) in settings without stochastic frame-skipping, while performance degrades in frameskipping settings.

show abstract

“…As the value of the M ′ moves away from 0, the interpretability of the system decreases. The idea of using complexity as a proxy for interpretability was also proposed in [4], where the authors stated that the computational complexity of a model can be used as a metric of interpretability as it directly resembles the number of operations that must be interpreted by humans.…”

Section: Related Workmentioning

confidence: 99%

Interpretable pipelines with evolutionarily optimized modules for RL tasks with visual inputs

Custode,

Iacca

2022

Preprint

0

View full text Add to dashboard Cite

The importance of explainability in AI has become a pressing concern, for which several explainable AI (XAI) approaches have been recently proposed. However, most of the available XAI techniques are post-hoc methods, which however may be only partially reliable, as they do not reflect exactly the state of the original models. Thus, a more direct way for achieving XAI is through interpretable (also called glass-box) models. These models have been shown to obtain comparable (and, in some cases, better) performance with respect to black-boxes models in various tasks such as classification and reinforcement learning. However, they struggle when working with raw data, especially when the input dimensionality increases and the raw inputs alone do not give valuable insights on the decision-making process. Here, we propose to use end-to-end pipelines composed of multiple interpretable models co-optimized by means of evolutionary algorithms, that allows us to decompose the decision-making process into two parts: computing high-level features from raw data, and reasoning on the extracted high-level features. We test our approach in reinforcement learning environments from the Atari benchmark, where we obtain comparable results (with respect to black-box approaches) in settings without stochastic frame-skipping, while performance degrades in frameskipping settings.

show abstract

Model Interpretability through the Lens of Computational Complexity

Cited by 4 publications

References 25 publications

Framework for Evaluating Faithfulness of Local Explanations

Framework for Evaluating Faithfulness of Local Explanations

Interpretable pipelines with evolutionary optimized modules for reinforcement learning tasks with visual inputs

Interpretable pipelines with evolutionarily optimized modules for RL tasks with visual inputs

Contact Info

Product

Resources

About