Safe Learning of Lifted Action Models

Abstract

Creating a domain model, even for classical, domain-independent planning, is a notoriously hard knowledge-engineering task. A natural approach to solve this problem is to learn a domain model from observations. However, model learning approaches frequently do not provide safety guarantees: the learned model may assume actions are applicable when they are not, and may incorrectly capture actions' effects. This may result in generating plans that will fail when executed. In some domains such failures are not acceptable, due to the cost of failure or inability to replan online after failure. In such settings, all learning must be done offline, based on some observations collected, e.g., by some other agents or a human. Through this learning, the task is to generate a plan that is guaranteed to be successful. This is called the model-free planning problem. Prior work proposed an algorithm for solving the model-free planning problem in classical planning. However, they were limited to learning grounded domains, and thus they could not scale. We generalize this prior work and propose the first safe model-free planning algorithm for lifted domains. We prove the correctness of our approach, and provide a statistical analysis showing that the number of trajectories needed to solve future problems with high probability is linear in the potential size of the domain model. We also present experiments on twelve IPC domains showing that our approach is able to learn the real action model in all cases with at most two trajectories.

Learning Probably Approximately Complete and Safe Action Models for Stochastic Worlds

“…SAM learning works by initially assuming every action has all the literals as preconditions and none of the literals as effects, and then applying rules 1 and 3 above to remove preconditions and add effects as needed. In a classical planning setting where all actions are grounded, Juba et al (2021) proved that the action model M SAM created by SAM learning is: (1) safe, in the sense that every plan consistent with M SAM is also consistent in the real action model, (2) probably complete, in the sense that with high probability, for most solvable problems there exists a plan that solves them and is consistent with M SAM , given a number of trajectories that is polynomial in the number of fluents and actions. They also extended SAM learning to learn lifted action models and provided similar safety and completeness guarantees.…”

“…The Safe Action Model (SAM) learning algorithm (Juba, Le, and Stern 2021;Stern and Juba 2017) has safety and completeness guarantees similar to those specified above. However, it is designed for classical planning, where states are fully observable and actions have deterministic effects.…”

“…But, in order to establish that our model generalizes across goals, we consider a different learning paradigm. Following a recent approach to learning deterministic action models (Stern and Juba 2017;Juba, Le, and Stern 2021), we suppose that problems for a domain are sampled from a fixed distribution, and we are given a training set of trajectories executing policies aiming to solve those problems in the domain. We seek an action model that captures enough of the domain faithfully to ensure that (i) policies that can be executed in the model behave similarly in the real world (solution safety) and (ii) policies that attain similar rates of success on the problem distribution as the training policy distribution can be executed in the model (solution completeness).…”

“…Action models are notoriously hard to formally specify, even in such a simple planning language. This has motivated work on a number of methods for automatically learning these action models from examples (Yang, Wu, and Jiang 2007;Cresswell and Gregory 2011;Cresswell, McCluskey, and West 2013;Zhuo and Kambhampati 2013;Stern and Juba 2017;Aineto, Celorrio, and Onaindia 2019;Juba, Le, and Stern 2021).…”

Goal Recognition as Reinforcement Learning

“…A large body of work involves learning for planning domains (Zimmerman and Kambhampati 2003;Arora et al 2018). While some approaches learn action models from data, they do not link these action models to policies for reaching specific goals (Amir and Chang 2008;Amado et al 2019;Asai and Muise 2020;Juba, Le, and Stern 2021).…”