From FOND to Robust Probabilistic Planning: Computing Compact Policies that Bypass Avoidable Deadends

Rivas, Alberto; Muise, Christian; McIlraith, Sheila A.

doi:10.1609/icaps.v26i1.13773

Cited by 14 publications

(10 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…LAO* generalizes heuristic search to solving belief MDPs (Hansen and Zilberstein 2001). PO-PRP (Muise, Belle, and McIlraith 2014) and ProbPRP (Camacho, Muise, and McIlraith 2016) use a series of calls to classical planners to iteratively construct and refine a policy for planning in a partially observable environment. However, these two methods neither exploit clear preferences nor aim to provide any guarantee on the value of the computed policy.…”

Section: Related Workmentioning

confidence: 99%

Fast Bounded Suboptimal Probabilistic Planning with Clear Preferences on Missing Information

Chatterjee

Kusnur

Likhachev

2021

SOCS

View full text Add to dashboard Cite

In the real-world, robots must often plan despite the environment being partially known. This frequently necessitates planning under uncertainty over missing information about the environment. Unfortunately, the computational expense of such planning often precludes its scalability to real-world problems. The Probabilistic Planning with Clear Preferences (PPCP) framework focuses on a specific subset of such planning problems wherein there exist clear preferences over the actual values of missing information (Likhachev and Stenz 2009). PPCP exploits the existence and knowledge of these preferences to perform provably optimal planning via a series of deterministic A*-like searches over particular instantiations of the environment. Such decomposition leads to much better scalability with respect to both the size of a problem and the amount of missing information in it. The run-time of PPCP however is a function of the number of searches it has to run until convergence. In this paper, we make a key observation that the number of searches PPCP has to run can be dramatically decreased if each search computes a plan that minimizes the amount of missing information it relies upon. To that end, we introduce Fast-PPCP, a novel planning algorithm that computes a provably bounded suboptimal policy using significantly lesser number of searches than that required to find an optimal policy. We present Fast-PPCP with its theoretical analysis, compare with common alternative approaches to planning under uncertainty over missing information, and experimentally show that Fast-PPCP provides substantial gain in runtime over other approaches while incurring little loss in solution quality.

show abstract

Section: Related Workmentioning

confidence: 99%

Fast Bounded Suboptimal Probabilistic Planning with Clear Preferences on Missing Information

Chatterjee

Kusnur

Likhachev

2021

SOCS

View full text Add to dashboard Cite

show abstract

“…Far beyond the standard benchmarks in Table 1 (triangle-side length 20), VI on BS scales to side length 74 in both the original domain and the limited-budget version. For comparison, the hitherto best solver by far was Prob-PRP (Camacho et al 2016), which scales to side length 70 on the original domain, 6 and is optimal only for goal probability 1, i. e., in the presence of strong cyclic plans.…”

Section: Acyclic Planningmentioning

confidence: 99%

“…Kolobov et al (2012) and Teichteil (2012) consider objectives asking for the cheapest policy among those maximizing goal probability, also requiring FRET or VI. Other works addressing goal probability maximization (e. g. (Teichteil-Königsbuch, Kuter, and Infantes 2010;Camacho et al 2016)) do not aim at guaranteeing optimality. In summary, heuristic search for MaxProb is challenging, and has only been addressed by Kolobov et al (2011).…”

Section: Introductionmentioning

confidence: 99%

“…Amongst other things, we observe: substantial benefits of heuristic search, even with trivial initial estimates (+9% total coverage), more so with initial estimates based on dead-end detection (+12%); substantial benefits of early termination (e. g. for AtleastProb +8% with θ = 0.2 and +7% with θ = 0.9); and dramatic benefits of our FRET variant (+32%). Our state-space reduction method yields an optimal MaxProb solver that scales just as well in TriangleTireworld as the sub-optimal solver Prob-PRP (Muise, McIlraith, and Beck 2012;Camacho et al 2016) -yet not only for the standard version where the goal can be achieved with certainty, but also for the limited-budget version where that is not so.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Revisiting Goal Probability Analysis in Probabilistic Planning

Steinmetz

Hoffmann

Buffet

2016

ICAPS

View full text Add to dashboard Cite

Maximizing goal probability is an important objective in probabilistic planning, yet algorithms for its optimal solution are severely underexplored. There is scant evidence of what the empirical state of the art actually is. Focusing on heuristic search, we close this gap with a comprehensive empirical analysis of known and adapted algorithms. We explore both, the general case where there may be 0-reward cycles, and the practically relevant special case of acyclic planning, like planning with a limited action-cost budget. We consider three different algorithmic objectives. We design suitable termination criteria, search algorithm variants, dead-end pruning methods using classical planning heuristics, and node selection strategies. Our evaluation on more than 1000 benchmark instances from the IPPC, resource-constrained planning, and simulated penetration testing reveals the behavior of heuristic search, and exhibits several improvements to the state of the art.

show abstract

“…For SSPs with dead ends, some research has focused only on finding policies that maximize the probability of reaching a goal (MAXPROB criterion) (Kolobov et al 2011;Teichteil-Königsbuch, Kuter, and Infantes 2010;Camacho, Muise, and McIlraith 2016), while other approaches work with two criteria: maximizing the probability of reaching a goal and minimizing the average accumulated costs of reaching a goal (Teichteil-Königsbuch 2012;Kolobov, Mausam, and Weld 2012;Trevizan, Teichteil-Königsbuch, and Thiébaux 2017).…”

Section: Introductionmentioning

confidence: 99%

An Exact Algorithm to Make a Trade-Off between Cost and Probability in SSPs

Freire

Delgado

Reis

2019

ICAPS

View full text Add to dashboard Cite

In stochastic sequential decision problems, such as Stochastic Shortest Path Problems, the GUBS (Goal with UtilityBased Semantics) criterion considers a trade-off between probability-to-goal and cost-to-goal using a goal semantics based on Expected Utility Theory (EUT); in such a semantics, goal paths have priority over non-goal paths, but it implies neither the MAXPROB criterion nor the dual optimization criterion that finds the cheapest policy among the policies that maximize goal probability. Whereas evaluation criteria based on a sound theory such as EUT are desirable, optimal policies under GUBS are non-markovian. Non-markovian solutions are undesirable because there is not always a finite representation and even if it can be represented in a finite way, the representation may be too large to be stored. Here we define a special case of GUBS criterion that allows a finite representation to the optimal policy, the eGUBS criterion, where the cost utility function is exponencial. Considering this special case, we contribute with: (i) the proof that the eGUBSoptimal policy has a finite representation; (ii) the first exact algorithm to obtain finite optimal policies for the eGUBS criterion, and (iii) four strategies to find sub-optimal policies. We conduct experiments on one synthetic problem to evaluate each strategy. Although optimal solutions have a high memory cost, sub-optimal policies can save memory space with a small decrease in performance.

show abstract

From FOND to Robust Probabilistic Planning: Computing Compact Policies that Bypass Avoidable Deadends

Cited by 14 publications

References 12 publications

Fast Bounded Suboptimal Probabilistic Planning with Clear Preferences on Missing Information

Fast Bounded Suboptimal Probabilistic Planning with Clear Preferences on Missing Information

Revisiting Goal Probability Analysis in Probabilistic Planning

An Exact Algorithm to Make a Trade-Off between Cost and Probability in SSPs

Contact Info

Product

Resources

About