Explainable AI and Causal Understanding: Counterfactual Approaches Considered

Baron, Sam

doi:10.1007/s11023-023-09637-x

Cited by 16 publications

(4 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…She worries that her application was denied because of her race or her gender. Sam Baron (2023) recently proposed one way that the bank can provide an explanation for its AI's decisions. The bank can make a list of counterfactuals of the form "If Sally's income had been $50,000, then the AI would have accepted her application.…”

Section: Discussionmentioning

confidence: 99%

Untitled

2023

JGS

View full text Add to dashboard Cite

The potential implications of AlphaGo defeating a human player have not been fully discussed in the epistemological field. AlphaGo does not play in a nearly exhaustive way with a huge amount of computation, but has reached a higher level of "intuition" and "judgment", which humans once thought was a difficult area for computer languages to break through. This phenomenon prompts us to rethink the structure of Go knowledge.From the perspective of epistemology, what kind of knowledge of Go is indeed reliable? Based on this fundamental question, this paper attempts to analyze the structure of Go knowledge. The main thought processes in the game of Go can be summarized as "intuition", "calculation" and "judge-

show abstract

Section: Discussionmentioning

confidence: 99%

Untitled

2023

JGS

View full text Add to dashboard Cite

show abstract

“…We would like to know about at least some processes, mechanisms, constraints, or structural dependencies inside of the model, rather than merely probing the ML-model-as-black-box from the outside and post hoc. While XAI methods can yield information that seems plausible and physically meaningful, this information may be irrelevant with respect to how the model actually arrived at a given decision or prediction (Rudin 2019;Baron 2023). This, in turn, can undermine our trust in the model for future applications.…”

Section: Post-hoc Xai In Climate Science and Statistical Understandingmentioning

confidence: 99%

Moving beyond post-hoc XAI: Lessons learned from dynamical climate modeling

O'Loughlin,

Li,

O'Brien

2024

Preprint

View full text Add to dashboard Cite

Abstract. AI models are criticized as being black boxes, potentially subjecting climate science to greater uncertainty. Explainable artificial intelligence (XAI) has been proposed to probe AI models and increase trust. In this Perspective, we suggest that, in addition to using XAI methods, AI researchers in climate science can learn from past successes in the development of physics-based dynamical climate models. Dynamical models are complex but have gained trust because their successes and failures can be attributed to specific components or sub-models, such as when model bias is explained by pointing to a particular parameterization. We propose three types of understanding as a basis to evaluate trust in dynamical and AI models alike: (1) instrumental understanding, which is obtained when a model has passed a functional test; (2) statistical understanding, which is obtained when researchers can make sense of the modelling results using statistical techniques to identify input-output relationships; and (3) Component-level understanding, which refers to modelers’ ability to point to specific model components or parts in the model architecture as the culprit for erratic model behaviors or as the crucial reason why the model functions well. We demonstrate how component-level understanding has been sought and achieved via climate model intercomparison projects over the past several decades. Such component-level of understanding routinely leads to model improvements and may also serve as a template for thinking about AI-driven climate science. Currently, XAI methods can help explain the behaviors of AI models by focusing on the mapping between input and output, thereby increasing the statistical understanding of AI models. Yet, to further increase our understanding of AI models, we will have to build AI models that have interpretable components amenable to component-level understanding. We give recent examples from the AI climate science literature to highlight some recent, albeit limited, successes in achieving component-level understanding and thereby explaining model behaviour. The merit of such interpretable AI models is that they serve as a stronger basis for trust in climate modeling and, by extension, downstream uses of climate model data.

show abstract

“…8 Moreover, there are many systems that demonstrate emergent behaviour where parthood is appealed to for at least a partial explanation of that emergence. For example, it is used to in the explanation of emergent characteristics of social behaviour (Abdou & Gilbert, 2009;Epstein, 2002;Schelling, 1971); economic behaviour (Chen & Yeh, 2002;Dosi & Roventini, 2019); behaviours of biological systems (Vicsek et al, 1995;Winfree, 1967); and artificial neural networks, such as machine learning algorithms like Deep Learning (Baron, 2023;Gupta & Jayannavar, 2021).…”

Section: Motivations For Mereological Models Of Spacetime Emergencementioning

confidence: 99%

Mereological Models of Spacetime Emergence

Pohlmann

2024

Philosophy Compass

View full text Add to dashboard Cite

Recent work in quantum gravity has prompted a re‐evaluation of the fundamental nature of spacetime. Spacetime is potentially emergent from non‐spatiotemporal entities posited by a theory of quantum gravity. Recent efforts have sought to interpret the relationship between spacetime and the fundamental entities through a mereological framework. These frameworks propose that spacetime can be conceived as either having non‐spatiotemporal entities as its constituents or being a constituent part of a non‐spatiotemporal structure. I present a roadmap for those interested in exploring the role of composition in understanding the emergence of spacetime. I establish a taxonomy based on four crucial parameters that should be considered when constructing potential mereological models. Subsequently, I connect these models to a range of perspectives found in current literature, with the aim of pinpointing areas that require further exploration. Finally, I identify three potential challenges facing mereological models of spacetime emergence, rooted in issues of mereological harmony, the distinction between continuous and discrete spacetime, and the implications of quantum superposition.

show abstract

Explainable AI and Causal Understanding: Counterfactual Approaches Considered

Cited by 16 publications

References 42 publications

Untitled

Untitled

Moving beyond post-hoc XAI: Lessons learned from dynamical climate modeling

Mereological Models of Spacetime Emergence

Contact Info

Product

Resources

About