Advanced planning for autonomous vehicles using reinforcement learning and deep inverse reinforcement learning

You, Changxi; Lǚ, Jianbo; Filev, Dimitar; Tsiotras, Panagiotis

doi:10.1016/j.robot.2019.01.003

Cited by 225 publications

(85 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The algorithms [130] without any prior knowledge and starting to play random gains, in 24 hours, achieved super-human level performance in chess, shogi as well as Go and convincingly defeated a work-champion program in each case. Since then the algorithm has been applied to solve more engineering problems like advanced planning of autonomous vehicles [131], lung cancer detection in medical treatment [132], smart agriculture [133], UAV cluster task scheduling [134], chatbots [135], autonomous building energy assessment [136].…”

Section: B Data-driven Modelingmentioning

confidence: 99%

Digital Twin: Values, Challenges and Enablers From a Modeling Perspective

2020

View full text Add to dashboard Cite

A digital twin can be defined as an adaptive model of a complex physical system. Recent advances in computational pipelines, multiphysics solvers, artificial intelligence, big data cybernetics, data processing and management tools bring the promise of digital twins and their impact on society closer to reality. Digital twinning is now an important and emerging trend in many applications. Also referred to as a computational megamodel, device shadow, mirrored system, avatar or a synchronized virtual prototype, there can be no doubt that a digital twin plays a transformative role not only in how we design and operate cyber-physical intelligent systems, but also in how we advance the modularity of multi-disciplinary systems to tackle fundamental barriers not addressed by the current, evolutionary modeling practices. In this work, we review the recent status of methodologies and techniques related to the construction of digital twins. Our aim is to provide a detailed coverage of the current challenges and enabling technologies along with recommendations and reflections for various stakeholders.

show abstract

Section: B Data-driven Modelingmentioning

confidence: 99%

Digital Twin: Values, Challenges and Enablers From a Modeling Perspective

2020

View full text Add to dashboard Cite

show abstract

“…where r t s is the safety penalty, r t e is the reward for agent's driving efficiency, r t t is the penalty for deviation from task's target and v i is the velocity of the i's traffic participant. As frequent lane changes can cause danger in traffic flow, a small penalty is set in r t s of (14) to discourage unnecessary lane changes. A reasonable agent drives as fast as possible within the speed limit of a lane.…”

Section: Rewards Designmentioning

confidence: 99%

“…To learn a simple and explainable driving policy, either states or actions or both in the discrete semantic form were used [11][12][13][14]. States can be "close to the front vehicle" and "far to the front vehicle" and the actions can be overtake, left change, give way and accelerate, respectively.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep, Consistent Behavioral Decision Making with Planning Features for Autonomous Vehicles

Qian

Zeng

2019

Electronics

View full text Add to dashboard Cite

Autonomous driving promises to be the main trend in the future intelligent transportation systems due to its potentiality for energy saving, and traffic and safety improvements. However, traditional autonomous vehicles’ behavioral decisions face consistency issues between behavioral decision and trajectory planning and shows a strong dependence on the human experience. In this paper, we present a planning-feature-based deep behavior decision method (PFBD) for autonomous driving in complex, dynamic traffic. We used a deep reinforcement learning (DRL) learning framework with the twin delayed deep deterministic policy gradient algorithm (TD3) to exploit the optimal policy. We took into account the features of topological routes in the decision making of autonomous vehicles, through which consistency between decision making and path planning layers can be guaranteed. Specifically, the features of a route extracted from path planning space are shared as the input states for the behavioral decision. The actor-network learns a near-optimal policy from the feasible and safe candidate emulated routes. Simulation tests on three typical scenarios have been performed to demonstrate the performance of the learning policy, including the comparison with a traditional rule-based expert algorithm and the comparison with the policy considering partial information of a contour. The results show that the proposed approach can achieve better decisions. Real-time test on an HQ3 (HongQi the third ) autonomous vehicle also validated the effectiveness of PFBD.

show abstract

“…Although there exist several optimal control solutions to the problem of overtaking maneuvers [1], [2], [3], [4], machine-learning-based methods have also been successfully applied. A reinforcement-learning-based overtaking control strategy is proposed in [5], [6]. In [7] a Q-learning strategy is used in the design of driving algorithms for multi-lane environments.…”

Section: Introductionmentioning

confidence: 99%

Performance Guarantees on Machine-Learning-based Overtaking Strategies for Autonomous Vehicles

Németh

Hegedüs

Gáspár

2020

2020 European Control Conference (ECC)

View full text Add to dashboard Cite

The control of autonomous vehicles in overtaking scenarios is an important challenge, in which an autonomous vehicle in a multiple vehicle environment must be safely driven. Due to the complexity of vehicle scenarios, several machinelearning-based design strategies have been developed, which provide outstanding results. However, in most of these methods it is difficult to provide a theoretical guarantee on the most important performance of the overtaking strategy, i.e., the avoidance of collisions with the surrounding vehicles. This paper proposes a design architecture with which this performance can be guaranteed. The method is based on the robust control framework and it is independent from the structure of the machine-learning-based agent. The effectiveness of the method is illustrated through simulation examples.

show abstract

Advanced planning for autonomous vehicles using reinforcement learning and deep inverse reinforcement learning

Cited by 225 publications

References 39 publications

Digital Twin: Values, Challenges and Enablers From a Modeling Perspective

Digital Twin: Values, Challenges and Enablers From a Modeling Perspective

Deep, Consistent Behavioral Decision Making with Planning Features for Autonomous Vehicles

Performance Guarantees on Machine-Learning-based Overtaking Strategies for Autonomous Vehicles

Contact Info

Product

Resources

About