Natural language understanding for logical games

Abstract

We developed a system able to automatically solve logical puzzles in natural language. Our solution is composed by a parser and an inference module. The parser translates the text into first order logic (FOL), while the MACE4 model finder is used to compute the models of the given FOL theory. We also empower our software agent with the capability to provide Yes/No answers to natural language questions related to each puzzle. Moreover, in line with Explainalbe Artificial Intelligence (XAI), the agent can back its answer, providing a graphical representation of the proof. The advantage of using reasoning for Natural Language Understanding (NLU) instead of Machine learning is that the user can obtain an explanation of the reasoning chain. We illustrate how the system performs on various types of natural language puzzles, including 382 knights and knaves puzzles. These features together with the overall performance rate of 80.89% makes the proposed solution an improvement upon similar solvers for natural language understanding in the puzzles domain.

A Puzzle-Based Dataset for Natural Language Inference

“…For the human agent, one can browse the 140 puzzles from the TPTP collection or the 144 puzzles modelled in FOL by Groza (2021). For the software agent, several puzzle solvers have been proposed (Lev et al (2004), Milicevic et al (2012, Bogaerts et al (2020), De Cat et al (2018, Jabrayilzade and Tekir (2020), Mitra and Baral (2015), Groza and Nitu (2021)). Lev et al (2004) have proposed a solution based on grammar rules, FOL, and model builders.…”

“…The target language is answer set programming, based on which 71 out of 100 puzzles have been solved. Groza and Nitu (2021) have also used grammar rules and named entity recognition to obtain a theory in FOL, which was given to Prover9 theorem prover. The grammar rules were manually created by analysing 43 puzzles for identifying the recurrent predicates, and then tested the resulted grammar on 331 puzzles.…”