DESPOT: Online POMDP Planning with Regularization

Ye, Nan; Somani, Adhiraj; Hsu, David; Lee, Wee Sun

doi:10.1613/jair.5328

Cited by 141 publications

(134 citation statements)

References 23 publications

Supporting

Mentioning

134

Contrasting

Order By: Relevance

“…POMDP planning suffers from the well-known "curse of dimensionality" and "curse of history" [8]: the complexity of planning grows exponentially with the size of the state space and the planning horizon. Two recent belief tree search algorithms, POMCP [9] and DESPOT [10] made online POMDP planning practical for real-world tasks. Both of them use Monte Carlo simulations to sample transitions and observations.…”

Section: A Online Pomdp Planningmentioning

confidence: 99%

LeTS-Drive: Driving in a Crowd by Learning from Tree Search

Cai

Luo

Saxena

et al. 2019

Robotics: Science and Systems XV

Self Cite

View full text Add to dashboard Cite

Autonomous driving in a crowded environment, e.g., a busy traffic intersection, is an unsolved challenge for robotics. The robot vehicle must contend with a dynamic and partially observable environment, noisy sensors, and many agents. A principled approach is to formalize it as a Partially Observable Markov Decision Process (POMDP) and solve it through online belief-tree search. To handle a large crowd and achieve realtime performance in this very challenging setting, we propose LeTS-Drive, which integrates online POMDP planning and deep learning. It consists of two phases. In the offline phase, we learn a policy and the corresponding value function by imitating the belief tree search. In the online phase, the learned policy and value function guide the belief tree search. LeTS-Drive leverages the robustness of planning and the runtime efficiency of learning to enhance the performance of both. Experimental results in simulation show that LeTS-Drive outperforms either planning or imitation learning alone and develops sophisticated driving skills.

show abstract

Section: A Online Pomdp Planningmentioning

confidence: 99%

LeTS-Drive: Driving in a Crowd by Learning from Tree Search

Cai

Luo

Saxena

et al. 2019

Robotics: Science and Systems XV

Self Cite

View full text Add to dashboard Cite

show abstract

“…There are two distinct approaches to planning under uncertainty: offline (e.g., [10,13,17,22]) and online (e.g., [19,21,25]). Offline planning leverages offline computation to reason about all future contingencies in advance and achieves faster execution time online.…”

Section: Background a Online Planning Under Uncertaintymentioning

confidence: 99%

“…For completeness, we provide a brief summary of the DESPOT algorithm. See [25] for details. To overcome the computational challenge of online planning under uncertainty, DESPOT samples a small finite set of K scenarios as representatives of the future.…”

Section: A Despotmentioning

confidence: 99%

“…DESPOT [25] is a state-of-the-art belief tree search algorithm for on planning under uncertainty. To overcome the computational challenge, DESPOT samples a set of "scenarios" and constructs incrementally-via heuristic tree search and Monte Carlo simulation-a sparse belief tree, which contains only branches reachable under the sampled scenarios (Fig.…”

Section: Introductionmentioning

confidence: 99%

“…1). The sparse tree is provably near-optimal [25], and DESPOT has shown strong performance in various robotic tasks, including autonomous driving [1] and manipulation [12].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

HyP-DESPOT: A Hybrid Parallel Algorithm for Online Planning under Uncertainty

Cai

Luo

Hsu

et al. 2018

Robotics: Science and Systems XIV

Self Cite

View full text Add to dashboard Cite

Abstract-Planning under uncertainty is critical for robust robot performance in uncertain, dynamic environments, but it incurs high computational cost. State-of-the-art online search algorithms, such as DESPOT, have vastly improved the computational efficiency of planning under uncertainty and made it a valuable tool for robotics in practice. This work takes one step further by leveraging both CPU and GPU parallelization in order to achieve near real-time online planning performance for complex tasks with large state, action, and observation spaces. Specifically, we propose Hybrid Parallel DESPOT (HyP-DESPOT), a massively parallel online planning algorithm that integrates CPU and GPU parallelism in a multi-level scheme. It performs parallel DESPOT tree search by simultaneously traversing multiple independent paths using multi-core CPUs and performs parallel Monte-Carlo simulations at the leaf nodes of the search tree using GPUs. Experimental results show that HyP-DESPOT speeds up online planning by up to several hundred times in several challenging robotic tasks in simulation, compared with the original DESPOT algorithm.

show abstract

TMPF: A Two‐Stage Merging Planning Framework for Dense Traffic

Chen

Cao

Guo

et al. 2023

Advanced Intelligent Systems

View full text Add to dashboard Cite

Planning for autonomous vehicles to merge into high‐density traffic flows within limited mileage is quite challenging. Specifically, the driving trajectory will inevitably have intersections with other vehicles whose driving intentions can't be directly observed. Herein, a two‐stage algorithm framework that is decomposed into the longitudinal and lateral planning processes for online merging planning is proposed. An improved particle filter is used to estimate the driving models of surrounding vehicles for predicting their future driving intentions. Based on Monte Carlo tree search (MCTS), different action spaces are evaluated for longitudinal merging gap selection and lateral interactive merging operation, while heuristic pruning is used to reduce the computation cost. Moreover, the coefficients related to the driving styles are introduced, and their influences on merging performance are analyzed. Finally, the proposed algorithm is implemented in a two‐lane simulation environment. The results show that the proposal has outperformed other baseline methods.

show abstract

DESPOT: Online POMDP Planning with Regularization

Cited by 141 publications

References 23 publications

LeTS-Drive: Driving in a Crowd by Learning from Tree Search

LeTS-Drive: Driving in a Crowd by Learning from Tree Search

HyP-DESPOT: A Hybrid Parallel Algorithm for Online Planning under Uncertainty

TMPF: A Two‐Stage Merging Planning Framework for Dense Traffic

Contact Info

Product

Resources

About