An Automatic Driving Control Method Based on Deep Deterministic Policy Gradient

Abstract

The traditional automatic driving behavior decision algorithm needs to manually set complex rules, resulting in long vehicle decision-making time, poor decision-making effect, and no adaptability to the new environment. As one of the main methods in the field of machine learning and intelligent control in recent years, reinforcement learning can learn reasonable and effective policies only by interacting with the environment. Firstly, this paper introduces the current research status of automatic driving technology and the current mainstream automatic driving control methods. Then, it analyzes the characteristics of convolutional neural network, reinforcement learning method ( Q -learning), and deep Q network (DQN) and deep deterministic policy gradient (DDPG). Compared with the DQN algorithm based on value function, the DDPG algorithm based on action policy can well solve the continuity problem of action space. Finally, the DDPG algorithm is used to solve the control problem of automatic driving. By designing a reasonable reward function, deep convolution network, and exploration policy, the intelligent vehicle can avoid obstacles and, finally, achieve the purpose of avoiding obstacles and running the whole process in a 2D environment.

Network Architecture for Optimizing Deep Deterministic Policy Gradient Algorithms

“…which normalizes gradients by using Z 1 and Z 2 . Otherwise, highly estimated errors would have been a certain case in a strict constraint if gradient normalization had not been used [34,35].…”

Dynamic Trajectory Planning and Optimization for Automated Driving on Ice and Snow Covered Road

“…Schaul uses deep reinforcement learning with a preferred experience playback mechanism instead of probability sampling, which improves the utilization of effective samples [14]. The machine learning algorithm requires a large sample space and is prone to errors when faced with new scenarios and problems [15]. However, trajectory planning on ISCR cannot provide a large number of samples and has low fault tolerance.…”

Learning to Drive in the NGSIM Simulator Using Proximal Policy Optimization

“…Te fourishing of machine learning enables using expressive models like neural networks to represent complex relationships like driving. Learning-based methods are attracting more attention in recent years [9,10]. Imitation learning (IL), one of the famous learning-based methods, has been applied by researchers to achieve AD [11,12].…”

“…Combined with deep learning, deep reinforcement learning (DRL) has been successfully applied to solve the game of GO [15], play Atari games [16], and accomplish loco-moto tasks [17]. In the application of applying DRL in AD, Zhang et al [9] used the deep deterministic policy gradient algorithm to achieve automatic driving. Te trained model can reach the defned goal and successfully avoid obstacles.…”