Deep Deterministic Policy Gradient Algorithm Based on Convolutional Block Attention for Autonomous Driving

Abstract

The research on autonomous driving based on deep reinforcement learning algorithms is a research hotspot. Traditional autonomous driving requires human involvement, and the autonomous driving algorithms based on supervised learning must be trained in advance using human experience. To deal with autonomous driving problems, this paper proposes an improved end-to-end deep deterministic policy gradient (DDPG) algorithm based on the convolutional block attention mechanism, and it is called multi-input attention prioritized deep deterministic policy gradient algorithm (MAPDDPG). Both the actor network and the critic network of the model have the same structure with symmetry. Meanwhile, the attention mechanism is introduced to help the vehicles focus on useful environmental information. The experiments are conducted in the open racing car simulator (TORCS)and the results of five experiment runs on the test tracks are averaged to obtain the final result. Compared with the state-of-the-art algorithm, the maximum reward increases from 62,207 to 116,347, and the average speed increases from 135 km/h to 193 km/h, while the number of success episodes to complete a circle increases from 96 to 147. Also, the variance of the distance from the vehicle to the center of the road is compared, and the result indicates that the variance of the DDPG is 0.6 m while that of the MAPDDPG is only 0.2 m. The above results indicate that the proposed MAPDDPG achieves excellent performance.

Towards Multi-Objective Object Push-Grasp Policy Based on Maximum Entropy Deep Reinforcement Learning under Sparse Rewards

“…The SumTree binary tree structure is employed to store samples in the prioritized experience replay buffer [28]. The samples with larger absolute TD-errors are more likely to be sampled, leading to faster convergence of the algorithm.…”

“…Therefore, based on the absolute value of the TD-error t δ for each sample, the priority of that sample is proportional to t δ . The SumTree binary tree structure is employed to store samples in the prioritized experience replay buffer [28]. The samples with larger absolute TD-errors are more likely to be sampled, leading to faster convergence of the algorithm.…”

Enhancing the landing guidance of a reusable launch vehicle by improving genetic algorithm-based deep reinforcement learning using Hybrid Deterministic-Stochastic algorithm

“…The researchers [ 29 ] established in their study that the policy’s performance is represented by this deterministic policy gradient. As seen in Fig 3 , the critic network determines the next state s’ of the actor network by assessing the state-action (s , a) pair value performance in order to maximize Q-value [ 30 ]. The output Q(s , a | Q) of the critic network is used to compute the definition of the critic loss function L., where N is minibatch size sample from the replay buffer, i -index refer the i -th sample, and y i is the temporal differencetarget.…”