Learning-Based Probabilistic LTL Motion Planning With Environment and Motion Uncertainties

Cai, Mingyu; Hao, Peng; Li, Zhijun; Kan, Zhen

doi:10.1109/tac.2020.3006967

Cited by 50 publications

(21 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where U tr π and U rec π are the expected return of states in transient and recurrent classes under policy π, respectively. In (13), P π (T , T ) ∈ R r×r is the probability transition matrix between states in T π , and…”

Section: Base Rewardmentioning

confidence: 99%

See 1 more Smart Citation

Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logic

Cai,

Hasanbeig,

Xiao

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

This paper investigates the motion planning of autonomous dynamical systems modeled by Markov decision processes (MDP) with unknown transition probabilities over continuous state and action spaces. Linear temporal logic (LTL) is used to specify high-level tasks over infinite horizon, which can be converted into a limit deterministic generalized Büchi automaton (LDGBA) with several accepting sets. The novelty is to design an embedded product MDP (EP-MDP) between the LDGBA and the MDP by incorporating a synchronous tracking-frontier function to record unvisited accepting sets of the automaton, and to facilitate the satisfaction of the accepting conditions. The proposed LDGBA-based reward shaping and discounting schemes for the model-free reinforcement learning (RL) only depend on the EP-MDP states and can overcome the issues of sparse rewards. Rigorous analysis shows that any RL method that optimizes the expected discounted return is guaranteed to find an optimal policy whose traces maximize the satisfaction probability. A modular deep deterministic policy gradient (DDPG) is then developed to generate such policies over continuous state and action spaces. The performance of our framework is evaluated via an array of OpenAI gym environments.

show abstract

Section: Base Rewardmentioning

confidence: 99%

“…However, scalability is a pressing issue for applying model-based approaches due to the need to store the learned model. On the other hand, by relaxing the need to construct an MDP model, model-free RL is recently adopted where appropriate reward shaping schemes are proposed [12][13][14][15][16][17][18][19][20].…”

Section: Introductionmentioning

confidence: 99%

Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logic

Cai,

Hasanbeig,

Xiao

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Puri elaborated on some of the drones available in the market that can be used for agricultural monitoring and observation to get better crop quality and to prevent various forms of damage to the fields (Puri et al, 2017). Currently, most human-robot skill systems focus on the service robot (Hu et al, 2019), manufacturing (Cai et al, 2020), medical rehabilitation (Su et al, 2020c) and robotic teleoperation (Yang et al, 2018c). The research of the humanrobot skill transfer system in agriculture still is one of the most challenging (Yingbai et al, 2020).…”

Section: Introductionmentioning

confidence: 99%

Human-robot skills transfer interface for UAV-based precision pesticide in dynamic environments

Zhou

Chen

et al. 2021

View full text Add to dashboard Cite

Purpose Nowadays, the global agricultural system is highly dependent on the widespread use of synthetic pesticides to control diseases, weeds and insects. The unmanned aerial vehicle (UAV) is deployed as a major part of integrated pest management in a precision agriculture system for accurately and cost-effectively distributing pesticides to resist crop diseases and insect pests. Design/methodology/approach With multimodal sensor fusion applying adaptive cubature Kalman filter, the position and velocity are enhanced for the correction and accuracy. A dynamic movement primitive is combined with the Gaussian mixture model to obtain numerous trajectories through the teaching of a demonstration. Further, to enhance the trajectory tracking accuracy under an uncertain environment of the spraying, a novel model reference adaptive sliding mode control approach is proposed for motion control. Findings Experimental studies have been carried out to test the ability of the proposed interface for the pesticides in the crop fields. The effectiveness of the proposed interface has been demonstrated by the experimental results. Originality/value To solve the path planning problem of a complex unstructured environment, a human-robot skills transfer interface is introduced for the UAV that is instructed to follow a trajectory demonstrated by a human teacher.

show abstract

“…On reinforcement learning with MDP, various methods exist to improve its behavior and performance for safety objectives (e.g. [24]), to deal with unknown stochastic behavior [5,25], and with linear-time logic specifications (e.g. [5,21,45]).…”

Section: Introductionmentioning

confidence: 99%

“…[24]), to deal with unknown stochastic behavior [5,25], and with linear-time logic specifications (e.g. [5,21,45]).…”

Section: Introductionmentioning

confidence: 99%

Learning optimal decisions for stochastic hybrid systems

Niehage

Hartmanns

Remke

2021

Proceedings of the 19th ACM-IEEE International Conference on Formal Methods and Models for System Design

View full text Add to dashboard Cite

We apply reinforcement learning to approximate the optimal probability that a stochastic hybrid system satisfies a temporal logic formula. We consider systems with (non)linear continuous dynamics, random events following general continuous probability distributions, and discrete nondeterministic choices. We present a discretized view of states to the learner, but simulate the continuous system. Once we have learned a near-optimal scheduler resolving the choices, we use statistical model checking to estimate its probability of satisfying the formula. We implemented the approach using Q-learning in the tools HYPEG and modes, which support Petri net-and hybrid automata-based models, respectively. Via two case studies, we show the feasibility of the approach, and compare its performance and effectiveness to existing analytical techniques for a linear model. We find that our new approach quickly finds near-optimal prophetic as well as non-prophetic schedulers, which maximize or minimize the probability that a specific signal temporal logic property is satisfied.

show abstract

Learning-Based Probabilistic LTL Motion Planning With Environment and Motion Uncertainties

Cited by 50 publications

References 28 publications

Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logic

Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logic

Human-robot skills transfer interface for UAV-based precision pesticide in dynamic environments

Learning optimal decisions for stochastic hybrid systems

Contact Info

Product

Resources

About