Tactical driving decisions of unmanned ground vehicles in complex highway environments: A deep reinforcement learning approach

Abstract

In this study, a deep reinforcement learning approach is proposed to handle tactical driving in complex highway traffic environments for unmanned ground vehicles. Tactical driving is a challenging topic for unmanned ground vehicles because of its interplay with routing decisions as well as real-time traffic dynamics. The core of our deep reinforcement learning approach is a deep Q-network that takes dynamic traffic information as input and outputs typical tactical driving decisions as action. The reward is designed with the consideration of successful highway exit, average traveling speed, and driving safety and comfort. In order to endow an unmanned ground vehicle with situational traffic information that is critical for tactical driving, the vehicle’s sensor information such as vehicle position and velocity are further augmented through the assessment of the ego-vehicle’s collision risk, potential field, and kinematics and used as input for the deep Q-network model. A convolutional neural network is built and fine-tuned to extract traffic features which facilitate the decision-making process of Q-learning. For model training and testing, a highway simulation platform is constructed with realistic parameter settings obtained from a real-world highway traffic dataset. The performance of the deep Q-network model is validated with extensive simulation experiments under different parameter settings such as traffic density and risk level. The results exhibit the important potentials of our deep Q-network model in learning challenging tactical driving decisions given multiple objectives and complex traffic environment.

A deep reinforcement learning-based approach for autonomous driving in highway on-ramp merge

“…In order to ensure the stability of the algorithm, the online parameters are cloned every N update episode to update the target network. 32…”

Decision making for highway complex scenario by improved safety field with learning process

“…Many researchers applied the RL for decision-making in specific scenarios, like the car-following, overtaking, or on-ramping scenario. [22][23][24][25][26][27][28] The deep deterministic policy gradient (DDPG) based car-flowing process is built in Zhu et al 22,29 Furthermore, the overtaking scenario is also learned by deep Q network, or Actor-critic method in Refs. 23,30,31 In these scenarios, the agent can be well trained to make decision.…”

“…In this part, the algorithm is further compared to the state-of-art algorithms like the pure safety field and the interaction-aware method, which is detailed introduced in Refs. 12,26,27 It is also carried out in the above complex traffic to validate the overall performance. The results are presented in Figures 15 to 17.…”

Predictive Control for Small Unmanned Ground Vehicles via a Multi-Dimensional Taylor Network

“…Recently, research on Unmanned Ground Vehicles (UGVs) has become a hot topic at home and abroad [1][2][3]. Its core technologies include environmental awareness, vehicle positioning, path planning, and path tracking control [4][5][6][7]. Path tracking control, as a key core issue of unmanned driving technology, has attracted much attention from scholars and it ensures that the car drives on the specified path [7][8][9].…”