Practical Open-Loop Optimistic Planning

Leurent, Edouard; Maillard, Odalric-Ambrym

doi:10.1007/978-3-030-46133-1_5

Cited by 6 publications

(8 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A first category of algorithms rely on optimistic planning [22], and require additional assumptions: a deterministic MDP [15], the open loop setting [2,21] in which policies are sequences of actions instead of state-action mappings (the two are equivalent in MDPs with deterministic transitions), or an MDP with known parameters [3]. For MDPs with stochastic and unknown transitions, polynomial sample complexities have been obtained for StOP [27], TrailBlazer [13] and SmoothCruiser [14] but the three algorithms suffer from numerical inefficiency, even for B < ∞.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Planning in Markov Decision Processes with Gap-Dependent Sample Complexity

Jonsson,

Kaufmann,

Ménard

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

We propose MDP-GapE, a new trajectory-based Monte-Carlo Tree Search algorithm for planning in a Markov Decision Process in which transitions have a finite support. We prove an upper bound on the number of calls to the generative models needed for MDP-GapE to identify a near-optimal action with high probability. This problemdependent sample complexity result is expressed in terms of the sub-optimality gaps of the state-action pairs that are visited during exploration. Our experiments reveal that MDP-GapE is also effective in practice, in contrast with other algorithms with sample complexity guarantees in the fixed-confidence setting, that are mostly theoretical.Preprint. Under review.

show abstract

Section: Related Workmentioning

confidence: 99%

“…We compare MDP-GapE to three existing baselines: first, the KL-OLOP algorithm [21], which uses the same upper-confidence bounds on the rewards u t h and states values U t h as MDP-GapE, but is restricted to open-loop policies, i.e. sequences of actions only.…”

Section: Fixed-budget Evaluationmentioning

confidence: 99%

Planning in Markov Decision Processes with Gap-Dependent Sample Complexity

Jonsson,

Kaufmann,

Ménard

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The problem formalization for workings of vehicle kinematics, temporal abstraction, partial observability and reward hypothesis has been studied extensively as well [35]. Robust optimization planning has been studied is past for finite MDP systems with uncertain parameters [37]- [39]. And also has shown promising results under conservative driving behavior.…”

Section: Previous Workmentioning

confidence: 99%

Building Safer Autonomous Agents by Leveraging Risky Driving Behavior Knowledge

Rana,

Malhi

2021

Preprint

View full text Add to dashboard Cite

Simulation environments are good for learning different driving tasks like lane changing, parking or handling intersections etc. in an abstract manner. However, these simulation environments often restrict themselves to operate under conservative interactions behavior amongst different vehicles. But, as we know that the real driving tasks often involves very high risk scenarios where other drivers often don't behave in the expected sense. There can be many reasons for this behavior like being tired or inexperienced. The simulation environments doesn't take this information into account while training the navigation agent. Therefore, in this study we especially focus on systematically creating these risk prone scenarios with heavy traffic and unexpected random behavior for creating better model-free learning agents. We generate multiple autonomous driving scenarios by creating new custom Markov Decision Process (MDP) environment iterations in highway-env simulation package. The behavior policy is learnt by agents trained with the help from deep reinforcement learning models. Our behavior policy is deliberated to handle collisions and risky randomized driver behavior. We train model free learning agents with supplement information of risk prone driving scenarios and compare their performance with baseline agents. Finally, we casually measure the impact of adding these perturbations in the training process to precisely account for the performance improvement attained from utilizing the learnings from these scenarios.

show abstract

“…Over the past decade, deep learning based approaches have been utilized with satisfactory results in various fields such as fair data generation [1], anomaly detection [2], accident detection [3], scene classification [4], hyperspectral image classification [5] and optimal path planning [6]. In comparison to typical path planning algorithms, reinforcement learning (RL) based approaches got significant attention in the recent past due to the success of the deep learning and computer vision.…”

Section: Introductionmentioning

confidence: 99%

“…Later was extended [27] for supporting stochastic rewards and dynamics in a open-loop setting, i.e, sequence of actions. The preceding algorithm was further improved in [6]. The modified version constraints on upper-confidence which helps to improve the performance considerably.…”

Section: Introductionmentioning

confidence: 99%

A Reinforcement Learning Based Path Planning Approach in 3d Environment

Kulathunga

Trihleb

2021

Mathematical Methods in Technologies and Technics

View full text Add to dashboard Cite

The optimal trajectory planning involves obstacles avoidance in which path planning is the key to success in optimal trajectory planning. Due the computational demands most of the path planning algorithms can not be employed for real-time based applications. Model-based Reinforcement Learning approaches for path planning got certain success in the recent past. Yet, most of such approaches do not have deterministic output due the nature of the those approaches. We analyzed a several types of reinforcement learning based approaches for path planning. One of them is deterministic tree-based approach and other two approaches are based on Q-learning and approximate policy gradient, respectively. We tested preceding approaches on two different type of simulators. Each of which consists of a set of random obstacles which could be changed or moved dynamically. After analysing the result and computation time, we concluded that deterministic tree search approach provides highly accurate result. However, computational time is considerably higher than the other two approaches. Finally, the comparative results are provided in terms of accuracy and computational time as the evidence.

show abstract

Practical Open-Loop Optimistic Planning

Cited by 6 publications

References 9 publications

Planning in Markov Decision Processes with Gap-Dependent Sample Complexity

Planning in Markov Decision Processes with Gap-Dependent Sample Complexity

Building Safer Autonomous Agents by Leveraging Risky Driving Behavior Knowledge

A Reinforcement Learning Based Path Planning Approach in 3d Environment

Contact Info

Product

Resources

About