Optimal continuous state POMDP planning with semantic observations

Burks, Luke; Ahmed, Nisar

doi:10.1109/cdc.2017.8263866

Cited by 7 publications

(7 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This work significantly extends the theory and results presented by the authors in [6]. Specifically, this paper provides arXiv:1807.08229v1 [cs.AI] 22 Jul 2018 a more detailed and generalized derivation of the VB GM policy approximation approach for linear-Gaussian dynamical systems, and provides a more rigorous analysis of the clustering-based GM condensation algorithm.…”

Section: Introductionsupporting

confidence: 74%

Optimal Continuous State POMDP Planning With Semantic Observations: A Variational Approach

2019

Self Cite

View full text Add to dashboard Cite

This work develops novel strategies for optimal planning with semantic observations using continuous state Partially Observable Markov Decision Processes (CPOMDPs). Two major innovations are presented in relation to Gaussian mixture (GM) CPOMDP policy approximation methods. While existing methods have many theoretically nice properties, they are hampered by the inability to efficiently represent and reason over hybrid continuous-discrete probabilistic models. The first major innovation is the derivation of closed-form variational Bayes GM approximations of Point-Based Value Iteration Bellman policy backups, using softmax models of continuous-discrete semantic observation probabilities. A key benefit of this approach is that dynamic decision-making tasks can be performed with complex non-Gaussian uncertainties, while also exploiting continuous dynamic state space models (thus avoiding cumbersome and costly discretization). The second major innovation is a new clusteringbased technique for mixture condensation that scales well to very large GM policy functions and belief functions. Simulation results for a target search and interception task with semantic observations show that the GM policies resulting from these innovations are more effective than those produced by other state of the art GM and Monte Carlo based policy approximations, but require significantly less modeling overhead and runtime cost. Additional results demonstrate the robustness of this approach to model errors.

show abstract

Section: Introductionsupporting

confidence: 74%

Optimal Continuous State POMDP Planning With Semantic Observations: A Variational Approach

2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…search and rescue or disaster relief. Building on the work here and in [7], we will investigate how semantic human sensor data can be actively leveraged for online interactive learning and planning, as well as online state estimation and perception.…”

Section: Discussionmentioning

confidence: 99%

“…The introduction of the CPOMDP method [15] showed that sets of Gaussian Mixture (GM) models can be used to approximate the policy over a continuous state space, while Switching-Mode POMDPs [16] further extended the CPOMDP framework to account for non-constant transition functions such as those caused by the presence of obstructions in the space. Finally, Variational Bayes POMDPs (VB-POMDPs) [7] were developed to handle non-Gaussian observation models in the form of softmax models, which easily model semantic observation statements and excel at parsing semantic input statements over a continuous space while significantly decreasing the computational cost of finding and implementing a policy.…”

Section: B Active Semantic Sensing For Planning Under Uncertaintymentioning

confidence: 99%

“…The full discrete observation space is given as O = O v × O q , with a set size |O| = 4. In this work continuous space observation models were constructed using softmax models as in [7].…”

Section: A Lower Level Cpomdpmentioning

confidence: 99%

“…Our technical approach builds on recent foundational work for efficiently finding policies based on continuous partially observable Markov decision process (CPOMDP) models [7]. While this CPOMDP approach provides several nice theoretical features for collaborative dynamic target search, we address some open issues that are crucial for real practical system deployment.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Closed-Loop Bayesian Semantic Data Fusion for Collaborative Human-Autonomy Target Search

Burks

Loefgren

Barbier

et al. 2018

2018 21st International Conference on Information Fusion (FUSION)

Self Cite

View full text Add to dashboard Cite

In search applications, autonomous unmanned vehicles must be able to efficiently reacquire and localize mobile targets that can remain out of view for long periods of time in large spaces. As such, all available information sources must be actively leveraged -including imprecise but readily available semantic observations provided by humans. To achieve this, this work develops and validates a novel collaborative human-machine sensing solution for dynamic target search. Our approach uses continuous partially observable Markov decision process (CPOMDP) planning to generate vehicle trajectories that optimally exploit imperfect detection data from onboard sensors, as well as semantic natural language observations that can be specifically requested from human sensors. The key innovation is a scalable hierarchical Gaussian mixture model formulation for efficiently solving CPOMDPs with semantic observations in continuous dynamic state spaces. The approach is demonstrated and validated with a real human-robot team engaged in dynamic indoor target search and capture scenarios on a custom testbed.

show abstract

Mori-zwanzig approach for belief abstraction with application to belief space planning

Hou,

Lin,

Zhou

et al. 2024

Auton Robot

View full text Add to dashboard Cite

We propose a learning-based method to extract symbolic representations of the belief state and its dynamics in order to solve planning problems in a continuous-state partially observable Markov decision processes (POMDP) problem. While existing approaches typically parameterize the continuous-state POMDP into a finite-dimensional Markovian model, they are unable to preserve fidelity of the abstracted model. To improve accuracy of the abstracted representation, we introduce a memory-dependent abstraction approach to mitigate the modeling error. The first major contribution of this paper is we propose a Neural Network based method to learn the non-Markovian transition model based on the Mori-Zwanzig (M-Z) formalism. Different from existing work in applying M-Z formalism to autonomous time-invariant systems, our approach is the first work generalizing the M-Z formalism to robotics, by addressing the non-Markovian modeling of the belief dynamics that is dependent on historical observations and actions. The second major contribution is we theoretically show that modeling the non-Markovian memory effect in the abstracted belief dynamics improves the modeling accuracy, which is the key benefit of the proposed algorithm. Simulation experiment of a belief space planning problem is provided to validate the performance of the proposed belief abstraction algorithms.

show abstract

Optimal continuous state POMDP planning with semantic observations

Cited by 7 publications

References 8 publications

Optimal Continuous State POMDP Planning With Semantic Observations: A Variational Approach

Optimal Continuous State POMDP Planning With Semantic Observations: A Variational Approach

Closed-Loop Bayesian Semantic Data Fusion for Collaborative Human-Autonomy Target Search

Mori-zwanzig approach for belief abstraction with application to belief space planning

Contact Info

Product

Resources

About