Towards the Role of Theory of Mind in Explanation

Shvo, Maayan; Klassen, Toryn Q.; McIlraith, Sheila A.

doi:10.1007/978-3-030-51924-7_5

Cited by 12 publications

(10 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A recent research direction that is closely related to the proposed notion of explanation is that by Shvo, Klassen, and McIlraith (2020), where they propose a general belief-based framework for generating explanations that employs epistemic state theory to capture the models of the explainer (agent in this paper) and the explainee (human user in this paper), and incorporates a belief revision operator to assimilate explanations into the explainee's epistemic states. A main difference with our proposed framework is that our framework restricts knowledge to be stored in logical formulae, while theirs considers epistemic states that can characterize different types of problems and have no such restriction.…”

Section: Some Further Discussionmentioning

confidence: 99%

A Logic-Based Explanation Generation Framework for Classical and Hybrid Planning Problems

Vasileiou

Yeoh²,

Son³

et al. 2022

jair

View full text Add to dashboard Cite

In human-aware planning systems, a planning agent might need to explain its plan to a human user when that plan appears to be non-feasible or sub-optimal. A popular approach, called model reconciliation, has been proposed as a way to bring the model of the human user closer to the agent’s model. To do so, the agent provides an explanation that can be used to update the model of human such that the agent’s plan is feasible or optimal to the human user. Existing approaches to solve this problem have been based on automated planning methods and have been limited to classical planning problems only. In this paper, we approach the model reconciliation problem from a different perspective, that of knowledge representation and reasoning, and demonstrate that our approach can be applied not only to classical planning problems but also hybrid systems planning problems with durative actions and events/processes. In particular, we propose a logic-based framework for explanation generation, where given a knowledge base KBa (of an agent) and a knowledge base KBh (of a human user), each encoding their knowledge of a planning problem, and that KBa entails a query q (e.g., that a proposed plan of the agent is valid), the goal is to identify an explanation ε ⊆ KBa such that when it is used to update KBh, then the updated KBh also entails q. More specifically, we make the following contributions in this paper: (1) We formally define the notion of logic-based explanations in the context of model reconciliation problems; (2) We introduce a number of cost functions that can be used to reflect preferences between explanations; (3) We present algorithms to compute explanations for both classical planning and hybrid systems planning problems; and (4) We empirically evaluate their performance on such problems. Our empirical results demonstrate that, on classical planning problems, our approach is faster than the state of the art when the explanations are long or when the size of the knowledge base is small (e.g., the plans to be explained are short). They also demonstrate that our approach is efficient for hybrid systems planning problems. Finally, we evaluate the real-world efficacy of explanations generated by our algorithms through a controlled human user study, where we develop a proof-of-concept visualization system and use it as a medium for explanation communication.

show abstract

Section: Some Further Discussionmentioning

confidence: 99%

A Logic-Based Explanation Generation Framework for Classical and Hybrid Planning Problems

Vasileiou

Yeoh²,

Son³

et al. 2022

jair

View full text Add to dashboard Cite

show abstract

“…In either case, we can predict that it will actively and intentionally seek behavior that aligns with its desires (Bennett and Maruyama, 2021 ). In this way, consistency in behaviors, helps develop accurate theory of mind (Shvo et al, 2020 ), and enables us to better interact, and establish efficient and effective communication (Rabinowitz et al, 2018 ), which are all crucial for collaboration with human teammates (Shergadwala and El-Nasr, 2021 ).…”

Section: Transparency and Trust In Human-agent Interaction Through Atommentioning

confidence: 99%

“…As described, these are based on internal models of contextually-relevant information and of prior interactions. ASI can use this to develop tailored interactions that make their intentions clear to users (Shvo et al, 2020 ). Agents with well-developed AToM are better able to communicate information that allows the user to understand why it took a certain action over another.…”

Section: Transparency and Trust In Human-agent Interaction Through Atommentioning

confidence: 99%

Supporting Artificial Social Intelligence With Theory of Mind

Williams

Fiore

Jentsch

2022

Front. Artif. Intell.

View full text Add to dashboard Cite

In this paper, we discuss the development of artificial theory of mind as foundational to an agent's ability to collaborate with human team members. Agents imbued with artificial social intelligence will require various capabilities to gather the social data needed to inform an artificial theory of mind of their human counterparts. We draw from social signals theorizing and discuss a framework to guide consideration of core features of artificial social intelligence. We discuss how human social intelligence, and the development of theory of mind, can contribute to the development of artificial social intelligence by forming a foundation on which to help agents model, interpret and predict the behaviors and mental states of humans to support human-agent interaction. Artificial social intelligence will need the processing capabilities to perceive, interpret, and generate combinations of social cues to operate within a human-agent team. Artificial Theory of Mind affords a structure by which a socially intelligent agent could be imbued with the ability to model their human counterparts and engage in effective human-agent interaction. Further, modeling Artificial Theory of Mind can be used by an ASI to support transparent communication with humans, improving trust in agents, so that they may better predict future system behavior based on their understanding of and support trust in artificial socially intelligent agents.

show abstract

“…Much research has been done on how it is possible to attribute beliefs and desires [4,5] in AI-based systems. However, we argue that the attribution of intentions, esp.…”

Section: Introductionmentioning

confidence: 99%

Why Interpreting Intent Is Key for Trustworthiness in the Age of Opaque Agents

Gimenez-Abalos,

Oliva-Felipe,

Vázquez-Salceda

et al. 2024

Preprint

View full text Add to dashboard Cite

This paper addresses the critical issue of trust in Artificial Intelligence systems, especially when users might find it challenging to comprehend the internal decision-making processes of such systems. A relevant topic of research in this respect is Theory of Mind, which involves attempting to understand these systems as if they possessed beliefs, desires, and intentions. We focus on the latter: intentions, and we examine how producing explanations based on them can improve \emph{understandability} while allowing for a better interpretation of how they align with human values. We also review some existing methods for identifying intentions in AI systems and we conclude with a discussion on possible future directions in this line of research.

show abstract

Towards the Role of Theory of Mind in Explanation

Cited by 12 publications

References 45 publications

A Logic-Based Explanation Generation Framework for Classical and Hybrid Planning Problems

A Logic-Based Explanation Generation Framework for Classical and Hybrid Planning Problems

Supporting Artificial Social Intelligence With Theory of Mind

Why Interpreting Intent Is Key for Trustworthiness in the Age of Opaque Agents

Contact Info

Product

Resources

About