Tackling Real-World Autonomous Driving using Deep Reinforcement Learning

Abstract

In the typical autonomous driving stack, planning and control systems represent two of the most crucial components in which data retrieved by sensors and processed by perception algorithms are used to implement a safe and comfortable self-driving behavior. In particular, the planning module predicts the path the autonomous car should follow taking the correct high-level maneuver, while control systems perform a sequence of low-level actions, controlling steering angle, throttle and brake. In this work, we propose a modelfree Deep Reinforcement Learning Planner training a neural network that predicts both acceleration and steering angle, thus obtaining a single module able to drive the vehicle using the data processed by localization and perception algorithms on board of the self-driving car. In particular, the system that was fully trained in simulation is able to drive smoothly and safely in obstacle-free environments both in simulation and in a real-world urban area of the city of Parma, proving that the system features good generalization capabilities also driving in those parts outside the training scenarios. Moreover, in order to deploy the system on board of the real self-driving car and to reduce the gap between simulated and real-world performances, we also develop a module represented by a tiny neural network able to reproduce the real vehicle dynamic behavior during the training in simulation.

Reinforcement Learning Review: Past Acts, Present Facts and Future Prospects

“…RL enables the systems to learn from the data obtained from the environment to make the correct decision. RL algorithms are used for the decision making and maneuver execution systems like lane change and keeping [76][77][78][79][80][81], overtaking maneuvers [82], intersection and roundabout handling [83][84]. According to [76], there are two crucial components of autonomous driving systems -planning and control systems.…”

“…RL algorithms are used for the decision making and maneuver execution systems like lane change and keeping [76][77][78][79][80][81], overtaking maneuvers [82], intersection and roundabout handling [83][84]. According to [76], there are two crucial components of autonomous driving systems -planning and control systems. The planning systems predict the path the self-driving car should take while the control systems are responsible for low-level actions like controlling steering angles, throttle, and break.…”

Towards Robust Decision-Making for Autonomous Highway Driving Based on Safe Reinforcement Learning

“…The development of RL in the field of autonomous driving has transitioned from foundational models to advanced algorithms capable of addressing complex and dynamic driving tasks. Early RL methods focused on simple control tasks [10], laying the foundation for more complex methods such as Deep Q-Network (DQN) for higher-dimensional state and action spaces [11], Deterministic Policy Gradient (DDPG) [12,13], Proximal Policy Optimization (PPO) [14], Trust Region Policy Optimization (TRPO) [14], and Asynchronous Advantage Actor Critic (A3C) [15], among others [16]. These methods have been used in the field of autonomous driving and have demonstrated good performance [15,17,18].…”

“…Early RL methods focused on simple control tasks [10], laying the foundation for more complex methods such as Deep Q-Network (DQN) for higher-dimensional state and action spaces [11], Deterministic Policy Gradient (DDPG) [12,13], Proximal Policy Optimization (PPO) [14], Trust Region Policy Optimization (TRPO) [14], and Asynchronous Advantage Actor Critic (A3C) [15], among others [16]. These methods have been used in the field of autonomous driving and have demonstrated good performance [15,17,18]. However, despite the remarkable performance of autonomous driving products showcased by companies such as Waymo, Baidu Apollo, and others in regular traffic, their safety reports have documented numerous emergency takeover incidents when faced with unknown or complex situations.…”