Learning Probably Approximately Complete and Safe Action Models for Stochastic Worlds

Abstract

We consider the problem of learning action models for planning in unknown stochastic environments that can be defined using the Probabilistic Planning Domain Description Language (PPDDL). As input, we are given a set of previously executed trajectories, and the main challenge is to learn an action model that has a similar goal achievement probability to the policies used to create these trajectories. To this end, we introduce a variant of PPDDL in which there is uncertainty about the transition probabilities, specified by an interval for each factor that contains the respective true transition probabilities. Then, we present SAM+, an algorithm that learns such an imprecise-PPDDL environment model. SAM+ has a polynomial time and sample complexity, and guarantees that with high probability, the true environment is indeed captured by the defined intervals. We prove that the action model SAM+ outputs has a goal achievement probability that is almost as good or better than that of the policies used to produced the training trajectories. Then, we show how to produce a PPDDL model based on this imprecise-PPDDL model that has similar properties.

Learning Safe Action Models with Partial Observability

“…FAMA works even if the observations given to it are partially observable. Algorithms from the SAM learning family (Stern and Juba 2017;Juba, Le, and Stern 2021;Juba and Stern 2022;Mordoch et al 2022) are different from other action model learning algorithms in that they guarantee that the action model they return is safe, in the sense that plans consistent with it are also consistent with the real, unknown action model. Most algorithms from this family have a tractable running time and reasonable sample complexity to ensure a probabilistic form of completeness, but rely on perfect observability of the given observations.…”

“…Plans generated with the learned model may not be executable or may fail to achieve their intended goals. SAM Learning (Stern and Juba 2017;Juba, Le, and Stern 2021;Juba and Stern 2022;Mordoch et al 2022) is a recently introduced family of learning algorithms that provide safety guarantees over the learned PDDL model: any plan generated with the model they return is guaranteed to be executable and achieve the intended goals. SAM Learning, however, is limited to learning from fully observed trajectories.…”

Safe Learning of PDDL Domains with Conditional Effects

“…The algorithms presented above learn action models that do not guarantee that the actions learned are applicable according to the agent's actual action model definition. Contrary to these algorithms, the SAM family of algorithms is designed to learn action models in a setting where execution failures must be avoided (Stern and Juba 2017;Juba, Le, and Stern 2021;Juba and Stern 2022). To this end, SAM generates a conservative action model.…”

“…We focus on such cases and aim to learn an action model that satisfies the strongest form of soundness: every plan generated using the learned model must be applicable and yield the same states as an unknown, accurate model. An action model that satisfies this requirement has been called safe (Juba, Le, and Stern 2021;Juba and Stern 2022;Mordoch, Stern, and Juba 2023). 1 We view this as a "safety" notion in part since it enables more conventional notions of safety to be enforced during planning, and provides assurance that they will carry over to the actual execution.…”

“…Algorithms from the Safe Action Model Learning (SAM) family (Stern and Juba 2017;Juba, Le, and Stern 2021;Juba and Stern 2022;Mordoch, Stern, and Juba 2023) address the challenge of learning safe action models under different sets of assumptions. However, these algorithms are not suitable for learning actions that may include conditional effects.…”