Engaging in Dialogue about an Agent’s Norms and Behaviors

Kasenberg, Daniel; Roque, Antonio; Thielstrom, Ravenna; Scheutz, Matthias

doi:10.18653/v1/w19-8406

Cited by 2 publications

(1 citation statement)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…How should an robot allude to its lack of environmental knowledge when explaining its decision-making process, and how can it encourage interactants to correct its errant world model? • The natural language capabilities themselves are limited to statements that can be converted into a particular subclass of VEL, described in Kasenberg et al [2019b]. • The architecture as presently constituted requires both utterances and follow-up questions to take very specific forms; it thus would not be particularly robust to interaction with humans not made aware of these forms.…”

Section: Explanation In Dialoguementioning

confidence: 99%

Explaining in Time

Arnold

Kasenberg

Scheutz

2021

J. Hum.-Robot Interact.

Self Cite

View full text Add to dashboard Cite

Explainability has emerged as a critical AI research objective, but the breadth of proposed methods and application domains suggest that criteria for explanation vary greatly. In particular, what counts as a good explanation, and what kinds of explanation are computationally feasible, has become trickier in light of oqaque “black box” systems such as deep neural networks. Explanation in such cases has drifted from what many philosophers stipulated as having to involve deductive and causal principles to mere “interpretation,” which approximates what happened in the target system to varying degrees. However, such post hoc constructed rationalizations are highly problematic for social robots that operate interactively in spaces shared with humans. For in such social contexts, explanations of behavior, and, in particular, justifications for violations of expected behavior, should make reference to socially accepted principles and norms. In this article, we show how a social robot’s actions can face explanatory demands for how it came to act on its decision, what goals, tasks, or purposes its design had those actions pursue and what norms or social constraints the system recognizes in the course of its action. As a result, we argue that explanations for social robots will need to be accurate representations of the system’s operation along causal, purposive, and justificatory lines. These explanations will need to generate appropriate references to principles and norms—explanations based on mere “interpretability” will ultimately fail to connect the robot’s behaviors to its appropriate determinants. We then lay out the foundations for a cognitive robotic architecture for HRI, together with particular component algorithms, for generating explanations and engaging in justificatory dialogues with human interactants. Such explanations track the robot’s actual decision-making and behavior, which themselves are determined by normative principles the robot can describe and use for justifications.

show abstract

Section: Explanation In Dialoguementioning

confidence: 99%

Explaining in Time

Arnold

Kasenberg

Scheutz

2021

J. Hum.-Robot Interact.

Self Cite

View full text Add to dashboard Cite

show abstract

Explainable reinforcement learning for broad-XAI: a conceptual framework and survey

Dazeley

Vamplew

Cruz

2023

Neural Comput & Applic

View full text Add to dashboard Cite

Broad-XAI moves away from interpreting individual decisions based on a single datum and aims to provide integrated explanations from multiple machine learning algorithms into a coherent explanation of an agent’s behaviour that is aligned to the communication needs of the explainee. Reinforcement Learning (RL) methods, we propose, provide a potential backbone for the cognitive model required for the development of Broad-XAI. RL represents a suite of approaches that have had increasing success in solving a range of sequential decision-making problems. However, these algorithms operate as black-box problem solvers, where they obfuscate their decision-making policy through a complex array of values and functions. EXplainable RL (XRL) aims to develop techniques to extract concepts from the agent’s: perception of the environment; intrinsic/extrinsic motivations/beliefs; Q-values, goals and objectives. This paper aims to introduce the Causal XRL Framework (CXF), that unifies the current XRL research and uses RL as a backbone to the development of Broad-XAI. CXF is designed to incorporate many standard RL extensions and integrated with external ontologies and communication facilities so that the agent can answer questions that explain outcomes its decisions. This paper aims to: establish XRL as a distinct branch of XAI; introduce a conceptual framework for XRL; review existing approaches explaining agent behaviour; and identify opportunities for future research. Finally, this paper discusses how additional information can be extracted and ultimately integrated into models of communication, facilitating the development of Broad-XAI.

show abstract

Engaging in Dialogue about an Agent’s Norms and Behaviors

Cited by 2 publications

References 11 publications

Explaining in Time

Explaining in Time

Explainable reinforcement learning for broad-XAI: a conceptual framework and survey

Contact Info

Product

Resources

About