A framework for step-wise explaining how to solve constraint satisfaction problems

Abstract

We explore the problem of step-wise explaining how to solve constraint satisfaction problems, with a use case on logic grid puzzles. More specifically, we study the problem of explaining the inference steps that one can take during propagation, in a way that is easy to interpret for a person. Thereby, we aim to give the constraint solver explainable agency, which can help in building trust in the solver by being able to understand and even learn from the explanations. The main challenge is that of finding a sequence of simple explanations, where each explanation should aim to be as cognitively easy as possible for a human to verify and understand. This contrasts with the arbitrary combination of facts and constraints that the solver may use when propagating. We propose the use of a cost function to quantify how simple an individual explanation of an inference step is, and identify the explanation-production problem of finding the best sequence of explanations of a CSP. Our approach is agnostic of the underlying constraint propagation mechanisms, and can provide explanations even for inference steps resulting from combinations of constraints. In case multiple constraints are involved, we also develop a mechanism that allows to break the most difficult steps up and thus gives the user the ability to zoom in on specific parts of the explanation. Our proposed algorithm iteratively constructs the explanation sequence by using an optimistic estimate of the cost function to guide the search for the best explanation at each step. Our experiments on logic grid puzzles show the feasibility of the approach in terms of the quality of the individual explanations and the resulting explanation sequences obtained.

Natural language understanding for logical games

“…Bogaerts et al have proposed a solver for logic grid puzzles which also makes use of QA and XAI [12]. De Cat et al employs an extension of FOL [13].…”

A Puzzle-Based Dataset for Natural Language Inference

“…For the human agent, one can browse the 140 puzzles from the TPTP collection or the 144 puzzles modelled in FOL by Groza (2021). For the software agent, several puzzle solvers have been proposed (Lev et al (2004), Milicevic et al (2012, Bogaerts et al (2020), De Cat et al (2018, Jabrayilzade and Tekir (2020), Mitra and Baral (2015), Groza and Nitu (2021)). Lev et al (2004) have proposed a solution based on grammar rules, FOL, and model builders.…”

“…The solver is designed around the Zebra puzzles and tested on a dataset of 68 puzzles. Bogaerts et al (2020) have proposed a solver for logic grid puzzles which also makes use of QA and XAI. De Cat et al (2018) have developed the IDP system based on an extension of FOL.…”