Hierarchical Evasive Path Planning Using Reinforcement Learning and Model Predictive Control

Abstract

Motion planning plays an essential role in designing self-driving functions for connected and autonomous vehicles. The methods need to provide a feasible trajectory for the vehicle to follow, fulfilling different requirements, such as safety, efficiency, and passenger comfort. In this area, algorithms must also meet strict real-time expectations, since, especially in an emergency, the decision time is limited, which raises a trade-off for the feasibility requirements. This article proposes a hierarchical path planning solution for evasive maneuvering, where a Twin Delayed DDPG reinforcement learning agent generates the parameters of a geometric path consisting of chlotoids and straight sections, and an underlying model predictive control loop fulfills the trajectory following tasks. The method is applied to the automotive double lane-change test, a common emergency situation, comparing its results with human drivers' performance using a dynamic simulation environment. Besides the test's standardized parameters, a broader range of topological layouts is chosen, both for the training and performance evaluation. The results show that the proposed method highly outperforms human drivers, especially in challenging situations, while meeting the computational requirements, as the pre-trained neural network and path generation algorithm can provide a solution in an instant, based on the experience gained during the training process.

Model Predictive Control of Internal Combustion Engines: A Review and Future Directions

“…For example, robotics, process control, and heating, ventilation, and air conditioning used (i) learning dynamic modeling for MPC by adjusting the model structure of MPC [192,[200][201][202][214][215][216], (ii) the controller design of MPC [193,[217][218][219][220], (iii) optimization of MPC solvers [196], (iv) imitation of MPC [195,195,221], and (v) MPC-based safe-learning of ML [194,222,223]. These methods seem promising for future implementation in ICE applications but must be comprehensively assessed.…”

Multi-directional-motion-coupling-based Extreme Motion Control of Distributed Drive Autonomous Vehicle

“…To adapt to the complex and changeable driving conditions of autonomous vehicles, considering the nonlinear dynamic characteristics, the path tracking control method based on dynamics has been widely studied 12,13 . The path tracking control method based on dynamics often takes the traditional linear two-degrees-of-freedom(2-DOF) model as the reference model 12,13 , and the control algorithms are PID algorithm 14 , model prediction algorithm 15 , sliding mode algorithm 16 , reinforcement learning control 17 .…”

Multidirectional motion coupling based extreme motion control of distributed drive autonomous vehicle

“…To adapt to the complex and changeable driving conditions of autonomous vehicles, considering the nonlinear dynamic characteristics, the path tracking control method based on dynamics has been widely studied 12 , 13 . The path tracking control method based on dynamics often takes the traditional linear two-degrees-of-freedom (2-DOF) model as the reference model 12 , 13 , and the control algorithms are PID algorithm 14 , model prediction algorithm 15 , sliding mode algorithm 16 , reinforcement learning control 17 .…”