Virtual to Real Reinforcement Learning for Autonomous Driving

Pan, Xinlei; You, Yurong; Wang, Ziyan; Lu, Cewu

doi:10.48550/arxiv.1704.03952

Cited by 85 publications

(72 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To extend this in the real-world, one option would be to leverage multiple pairs of physical protagonist and adversarial agents, which then update a global network. Alternatively, sim-to-real transfer, an active area of research [46]- [49], could be investigated to better leverage the faster training offered by simulators and minimising the amount of costly real-world training required.…”

Section: Discussionmentioning

confidence: 99%

ARC: Adversarially Robust Control Policies for Autonomous Vehicles

Kuutti¹,

Fallah²,

Bowden³

2021

Preprint

View full text Add to dashboard Cite

Deep neural networks have demonstrated their capability to learn control policies for a variety of tasks. However, these neural network-based policies have been shown to be susceptible to exploitation by adversarial agents. Therefore, there is a need to develop techniques to learn control policies that are robust against adversaries. We introduce Adversarially Robust Control (ARC), which trains the protagonist policy and the adversarial policy end-to-end on the same loss. The aim of the protagonist is to maximise this loss, whilst the adversary is attempting to minimise it. We demonstrate the proposed ARC training in a highway driving scenario, where the protagonist controls the follower vehicle whilst the adversary controls the lead vehicle. By training the protagonist against an ensemble of adversaries, it learns a significantly more robust control policy, which generalises to a variety of adversarial strategies. The approach is shown to reduce the amount of collisions against new adversaries by up to 90.25%, compared to the original policy. Moreover, by utilising an auxiliary distillation loss, we show that the fine-tuned control policy shows no drop in performance across its original training distribution.

show abstract

Section: Discussionmentioning

confidence: 99%

ARC: Adversarially Robust Control Policies for Autonomous Vehicles

Kuutti¹,

Fallah²,

Bowden³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In sequential decision making, the action of the previous step will affect the next step. Many problems can be represented in this form, such as game playing [15]- [17], autonomous driving [18]- [20], robot control [21], [22], recommended system [23] and trading [24]. Inspired by this work, we also adopt the classic architecture in which the policy network guides the MCTS.…”

Section: Related Workmentioning

confidence: 99%

Hierarchical Policy for Non-prehensile Multi-object Rearrangement with Deep Reinforcement Learning and Monte Carlo Tree Search

Bai¹,

Meng²,

Liu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Non-prehensile multi-object rearrangement is a robotic task of planning feasible paths and transferring multiple objects to their predefined target poses without grasping. It needs to consider how each object reaches the target and the order of object movement, which significantly deepens the complexity of the problem. To address these challenges, we propose a hierarchical policy to divide and conquer for non-prehensile multi-object rearrangement. In the high-level policy, guided by a designed policy network, the Monte Carlo Tree Search efficiently searches for the optimal rearrangement sequence among multiple objects, which benefits from imitation and reinforcement. In the low-level policy, the robot plans the paths according to the order of path primitives and manipulates the objects to approach the goal poses one by one. We verify through experiments that the proposed method can achieve a higher success rate, fewer steps, and shorter path length compared with the state-of-the-art.

show abstract

“…Until recently, the choice of an optimal action function was only solvable either in very specific settings where a closed form solution exists or where both the action and the state spaces are finite and rather small, see [WD92]. In the last decade, the combination of ideas from optimal control with the flexibility of deep learning has enabled astonishing progress in the context of self-driving cars, see [Pan+17], games, see [Mni+15] and robotics, see [Lil+15]. First for discrete action spaces, see [Mni+15], and even more recently for continuous and potentially high dimensional action spaces.…”

Section: Differentiable Reinforcement Learning 21 the Reinforcement L...mentioning

confidence: 99%

“…Its combination with deep learning, see [LBH15], called deep reinforcement learning (DRL), see [Fra+18], has recently had a huge success in addressing sequential decision problems. For example, it outperforms the best humans at many games, see [Mni+15], and has state of the art applications in many real life applications such as robotics, see [Lil+15], and self-driving cars, see [Pan+17].…”

Section: Introductionmentioning

confidence: 99%

Deep differentiable reinforcement learning and optimal trading

Jaisson¹

2021

Preprint

View full text Add to dashboard Cite

In this article we introduce the differentiable reinforcement learning framework. It is based on the fact that in many reinforcement learning applications, the environment reward and transition functions are not black boxes but known differentiable functions. Incorporating deep learning in this framework we find more accurate and stable solutions than more generic actor critic algorithms. We apply this deep differentiable reinforcement learning (DDRL) algorithm to the problem of optimal trading strategies in various environments where the market dynamics are known. Thanks to the stability of this method, we are able to efficiently find optimal strategies for complex multi-scale market models and for a wide range of environment parameters. This makes it applicable to real life financial signals and portfolio optimization where the expected return has multiple time scales. In the case of a slow and a fast alpha signal, we find that the optimal trading strategy consists in using the fast signal to time the trades associated to the slow signal.

show abstract

Virtual to Real Reinforcement Learning for Autonomous Driving

Cited by 85 publications

References 13 publications

ARC: Adversarially Robust Control Policies for Autonomous Vehicles

ARC: Adversarially Robust Control Policies for Autonomous Vehicles

Hierarchical Policy for Non-prehensile Multi-object Rearrangement with Deep Reinforcement Learning and Monte Carlo Tree Search

Deep differentiable reinforcement learning and optimal trading

Contact Info

Product

Resources

About