Contrastive Explanations of Plans through Model Restrictions

Abstract

In automated planning, the need for explanations arises when there is a mismatch between a proposed plan and the user’s expectation. We frame Explainable AI Planning as an iterative plan exploration process, in which the user asks a succession of contrastive questions that lead to the generation and solution of hypothetical planning problems that are restrictions of the original problem. The object of the exploration is for the user to understand the constraints that govern the original plan and, ultimately, to arrive at a satisfactory plan. We present the results of a user study that demonstrates that when users ask questions about plans, those questions are usually contrastive, i.e. “why A rather than B?”. We use the data from this study to construct a taxonomy of user questions that often arise during plan exploration. Our approach to iterative plan exploration is a process of successive model restriction. Each contrastive user question imposes a set of constraints on the planning problem, leading to the construction of a new hypothetical planning problem as a restriction of the original. Solving this restricted problem results in a plan that can be compared with the original plan, admitting a contrastive explanation. We formally define model-based compilations in PDDL2.1 for each type of constraint derived from a contrastive user question in the taxonomy, and empirically evaluate the compilations in terms of computational complexity. The compilations were implemented as part of an explanation framework supporting iterative model restriction. We demonstrate its benefits in a second user study.

Causal-Based Approaches to Explain and Learn from Self-Extension—A Review

“…Moreover, the latter allows each action to be available for execution as soon as it is generated, which in turn allows concurrent planning and execution. PDDL is used, for example, for the proposals of Krarup et al [46] or Lindner and Olz [16], which are cited in Table 1, but also for Sreedharan et al [55] or Stulp et al [56]. In general, the coding of preconditions and postconditions used in PDDL constitutes a causal model that can be quite complex and that can allow work on reconciliation between the robot and person models [55], or on post hoc explanation [46] (counterfactuals, i.e., an explanation that describes that the output of the model will be y ′ instead of y, thereby changing, without this change occurring in the real world, the behaviour or inputs x to x ′ ) [57].…”

“…The aim of providing explanations should be to improve these metrics. The topic is addressed in detail in Krarup et al [46]'s work.…”

“…To try to find out where the difference between the plans lies, the person expresses his or her doubts with questions. However, several studies [14,43,46] have shown that, when people ask questions about plans, those questions are contrastive, i.e., "Why did you do this instead of that?". The difference between this question and the question "Why do you do this?"…”

Logic-Based Explainability in Machine Learning

“…Although at present ML model explainability of ML models is the most studied theme in the general field of explainability, it is also the case that explainability has been studied in AI for decades [11-13, 96, 103, 104, 113, 250, 276, 278, 292, 293], with a renewed interest in recent years. For example, explanations have recently been studied in AI planning [65,94,95,111,142,194,288,289,291,302], constraint satisfaction and problem solving [54,99,115,135,289], among other examples [290]. Furthermore, there is some agreement that regulations like EU's General Data Protection Regulation (GDPR) [100] effectively impose the obligation of explanations for any sort of algorithmic decision making [129,188].…”

Evaluating Plan-Property Dependencies: A Web-Based Platform and User Study

“…Other XAIP user studies use one domain(Chakraborti et al 2019b;Chakraborti and Kambhampati 2019;Sreedharan et al 2019aLindsay et al 2020;Das, Banerjee, and Chernova 2021), two domains(Sreedharan et al 2019b;Sreedharan, Srivastava, and Kambhampati 2020) or four domains(Krarup et al 2021). …”