Deep Deterministic Policy Gradient Algorithm based Lateral and Longitudinal Control for Autonomous Driving

Zhu, Guangxi; Pei, Chunmei; Jiang, Ding De; Shi, Juntai

doi:10.1109/icmcce51767.2020.00163

Cited by 4 publications

(4 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The proposed work established the exact mapping using the DDPG algorithm on TORCS simulation software. Gongsheng et al 2020 [42] proposed a vehicle control algorithm designed on a TORCS simulator combined with DDPG. Combined with an actor-critic algorithm, experience playback, and a separate target network, DDPG has more robust effects.…”

Section: B Ddpg Based Papersmentioning

confidence: 99%

“…Hoel et al [40] MTCS-NN Proposed a combined method that uses MTCS and neural networks for planning and tactical decision making based on AlphaGo Zero algorithm. [41] DDPG algorithm implementation of a single agent for lane keeping scenario on TORCS Gongsheng et al [42] Proposed a vehicle control algorithm designed on a TORCS simulator combined with DDPG.…”

Section: Kartikeyan Et Al [20] Dqnmentioning

confidence: 99%

See 1 more Smart Citation

An Empirical Study of DDPG and PPO-Based Reinforcement Learning Algorithms for Autonomous Driving

Siboo,

Bhattacharyya,

Naveen Raj

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Autonomous vehicles mitigate road accidents and provide safe transportation with a smooth traffic flow. They are expected to greatly improve the quality of the elderly or people with impairments by improving their mobility due to the ease of access to transportation. Autonomous vehicles sense the driving environment and navigate through it without human intervention. And, Deep Reinforcement Learning (DRL) is witnessed as a powerful machine learning solution to address a sequential decision problem in autonomous vehicles. The detailed state-of-the-art work in autonomous vehicles using DRL algorithms along with future research directions is discussed. Due to the high dimensional action space, two continuous action space DRL algorithms: Deterministic Policy Gradient (DDPG) and Proximal Policy Optimization (PPO) are chosen to address the complex autonomous driving problem. The proposed DDPG and PPO based decision-making models are trained and tested using the TORC simulator. Both the algorithms are trained for the same number of episodes for lane keeping as well as multi-agent collision avoidance scenarios. To the best of our knowledge, this is the first paper to present the comparative performance analysis of these two algorithms, and DDPG is found to perform better in terms of higher reward and faster convergence than PPO. Hence, DDPG is a suitable option in the design of a decision model for autonomous driving.

show abstract

Section: B Ddpg Based Papersmentioning

confidence: 99%

Section: Kartikeyan Et Al [20] Dqnmentioning

confidence: 99%

An Empirical Study of DDPG and PPO-Based Reinforcement Learning Algorithms for Autonomous Driving

Siboo,

Bhattacharyya,

Naveen Raj

et al. 2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…(Pérez et al, 2010) propone un sistema de control longitudinal de un vehículo ajustado mediante técnicas neurodifusas. Los autores en (Gongsheng, et al, 2020) se basan en escenas de carretera reales haciendo uso de un simulador de conducción autónoma (AirSim), utilizando técnicas de aprendizaje profundo para realizar una conducción autónoma. En (Berahman et al,.…”

Section: Introductionunclassified

Control longitudinal de un vehículo mediante aprendizaje por refuerzo profundo

Barreno Herrera,

Santos,

Romana

2023

XLIV Jornadas De Automática: Libro De Actas: Universidad De Zaragoza, Escuela De Ingeniería Y Arquitectura, 6, 7 Y 8 De Septiem

View full text Add to dashboard Cite

Este artículo presenta un sistema inteligente para el control longitudinal de un vehículo mediante aprendizaje por refuerzo profundo basado en la influencia de la curvatura de la carretera. El sistema inteligente consiste en un agente que usa el algoritmo de gradiente de política determinista profundo (DDPG) para controlar la velocidad. Para entrenar al agente del modelo, el efecto de la curvatura de la carretera se considera a través de la aceleración percibida obtenida a partir de la aceleración lateral y velocidad angular debida a la propia carretera. Los resultados del sistema inteligente son valores continuos de aceleración. El modelo propuesto ofrece resultados prometedores, lo que sugiere que este sistema inteligente puede ayudar al conductor y que el sistema de control del vehículo puede aplicarse a la conducción semiautónoma o autónoma haciendo que la conducción sea más segura y cómoda.

show abstract

“…For example, the Deterministic Policy Gradient (DPG) [20], whose policies are deterministic, requiring no sampling integration in the action space, greatly cutting the required sample data, thus improving the computational efficiency. The Deep Deterministic Policy Gradient (DDPG) algorithm combines the advantages of DPG and Deep Q Network (DQN), for which as a basis, the 'Actor-Critic' framework is introduced additionally to realize real-time tracking [21,22] and dynamic programming of autonomous driving [23][24][25], as well as some other dynamic problems. The traditional RL method is difficult to be used for multi-agent, because the policy of each agent is constantly changing during the training process, which would cause the environment to be unstable; apparently the policy learned in an unstable environment is meaningless.…”

Section: Introductionmentioning

confidence: 99%

A Multi-Agent Adaptive Co-Evolution Method in Dynamic Environments

Zhang

et al. 2023

Mathematics

View full text Add to dashboard Cite

It is challenging to ensure satisfying co-evolution efficiency for the multi-agents in dynamic environments since during Actor-Critic training there is a high probability of falling into local optimality, failing to adapt to the suddenly changed environment quickly. To solve this problem, this paper proposes a multi-agent adaptive co-evolution method in dynamic environments (ACE-D) based on the classical multi-agent reinforcement learning method MADDPG, which effectively realizes self-adaptive new environments and co-evolution in dynamic environments. First, an experience screening policy is introduced based on the MADDPG method to reduce the negative influence of original environment experience on exploring new environments. Then, an adaptive weighting policy is applied to the policy network, which accordingly generates benchmarks for varying environments and assigns higher weights to those policies that are more beneficial for new environments exploration, so that to save time while promoting adaptability of the agents. Finally, different types of dynamic environments with complexity at different levels are built to verify the co-evolutionary effects of the two policies separately and the ACE-D method comprehensively. The experimental results demonstrate that, compared with a range of other methods, the ACE-D method has obvious advantages helping multi-agent adapt to dynamic environments and preventing them from falling into local optima, with more than 25% improvement in stable reward and more than 23% improvement in training efficiency. The ACE-D method is valuable and commendable to promote the co-evolutionary effect of multi-agent in dynamic environments.

show abstract

Deep Deterministic Policy Gradient Algorithm based Lateral and Longitudinal Control for Autonomous Driving

Cited by 4 publications

References 8 publications

An Empirical Study of DDPG and PPO-Based Reinforcement Learning Algorithms for Autonomous Driving

An Empirical Study of DDPG and PPO-Based Reinforcement Learning Algorithms for Autonomous Driving

Control longitudinal de un vehículo mediante aprendizaje por refuerzo profundo

A Multi-Agent Adaptive Co-Evolution Method in Dynamic Environments

Contact Info

Product

Resources

About