A Deep Deterministic Policy Gradient Approach for Vehicle Speed Tracking Control With a Robotic Driver

Hao, Gaofeng; Fu, Zhuang; Feng, Xin; Gong, Zening; Chen, Peng; Wang, Dan; Weibin, Wang; Si, Yang

doi:10.1109/tase.2021.3088004

Cited by 21 publications

(4 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…DDPG algorithm has been successfully applied to Robotics and motion control problems [81,82]. DDPG can be used for a variety of tasks, such as manipulation, locomotion, and navigation [83].…”

Section: Summary Of Studies Classified As Roboticsmentioning

confidence: 99%

“…Methodology: The TD3 algorithm is used to reduce overestimation bias in Deep Q-Learning with discrete actions in an Actor-Critic domain setting. 4-Ant-legged robot (Hao et al, 2022) [82] Techniques: (DDPG), replay buffer optimization, and weighted training samples. Methodology: The method interferes with network exploration by utilizing the fundamental relationship between pedal opening and vehicle speed.…”

Section: Robotic Surgery Andmentioning

confidence: 99%

See 1 more Smart Citation

Deep Deterministic Policy Gradient Algorithm: A Systematic Review

Sumiea,

AbdulKadir,

Al-Selwi

et al. 2023

Preprint

View full text Add to dashboard Cite

Deep Reinforcement Learning (DRL) has gained significant adoption in diverse fields and applications, mainly due to its proficiency in resolving complicated decision-making problems in spaces with high-dimensional states and actions. Deep Deterministic Policy Gradient (DDPG) is a well-known DRL algorithm that adopts an actor-critic approach, synthesizing the advantages of value-based and policy-based reinforcement learning methods. The aim of this study is to provide a thorough examination of the latest developments, patterns, obstacles, and potential opportunities related to DDPG. A systematic search was conducted using relevant academic databases (Scopus, Web of Science, and ScienceDirect) to identify 85 relevant studies published in the last five years (2018-2023). We provide a comprehensive overview of the key concepts and components of DDPG, including its formulation, implementation, and training. Then, we highlight the various applications and domains of DDPG, including Autonomous Driving, Unmanned Aerial Vehicles, Resource Allocation, Communications and the Internet of Things, Robotics, and Finance. Additionally, we provide an in-depth comparison of DDPG with other DRL algorithms and traditional RL methods, highlighting its strengths and weaknesses. We believe that this review will be an essential resource for researchers, offering them valuable insights into the methods and techniques utilized in the field of DRL and DDPG.

show abstract

Section: Summary Of Studies Classified As Roboticsmentioning

confidence: 99%

Section: Robotic Surgery Andmentioning

confidence: 99%

Deep Deterministic Policy Gradient Algorithm: A Systematic Review

Sumiea,

AbdulKadir,

Al-Selwi

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Fujimoto et al (2018) contribute a mechanism that takes the minimum value between a pair of critics in the actor-critic algorithm of Silver et al (2014) to tackle the function approximation errors. The deterministic policy gradient theory has been widely applied in various fields, such as electricity market (Liang et al, 2020), vehicle speed tracking control (Hao et al, 2021), fuzzy PID controller (Shi et al, 2020), quadrotor control (Wang et al, 2020), energy efficiency (Zhang et al, 2020), and autonomous underwater vehicles (Sun et al, 2020;Wu et al, 2022). However, to our best knowledge, it has never been employed in the AOR task.…”

Section: Figurementioning

confidence: 99%

Viewpoint planning with transition management for active object recognition

Sun

Zhao

et al. 2023

Front. Neurorobot.

View full text Add to dashboard Cite

Active object recognition (AOR) provides a paradigm where an agent can capture additional evidence by purposefully changing its viewpoint to improve the quality of recognition. One of the most concerned problems in AOR is viewpoint planning (VP) which refers to developing a policy to determine the next viewpoints of the agent. A research trend is to solve the VP problem with reinforcement learning, namely to use the viewpoint transitions explored by the agent to train the VP policy. However, most research discards the trained transitions, which may lead to an inefficient use of the explored transitions. To solve this challenge, we present a novel VP method with transition management based on reinforcement learning, which can reuse the explored viewpoint transitions. To be specific, a learning framework of the VP policy is first established via the deterministic policy gradient theory, which provides an opportunity to reuse the explored transitions. Then, we design a scheme of viewpoint transition management that can store the explored transitions and decide which transitions are used for the policy learning. Finally, within the framework, we develop an algorithm based on twin delayed deep deterministic policy gradient and the designed scheme to train the VP policy. Experiments on the public and challenging dataset GERMS show the effectiveness of our method in comparison with several competing approaches.

show abstract

“…In the field of RL, DDPG (Deep Deterministic Policy Gradient) combines the advantages of deep learning [59,60,61,62] and policy gradient methods to address reinforcement learning problems in continuous action spaces, making it highly favored by researchers [63,64,65,66,67]. Tao et al focused on utilizing parallel DDPG strategies for gait control.…”

Section: Introductionmentioning

confidence: 99%

Deep deterministic policy gradient with constraints for gait optimisation of biped robots

Liu,

Rong,

Neri

et al. 2024

ICA

View full text Add to dashboard Cite

In this paper, we propose a novel Reinforcement Learning (RL) algorithm for robotic motion control, that is, a constrained Deep Deterministic Policy Gradient (DDPG) deviation learning strategy to assist biped robots in walking safely and accurately. The previous research on this topic highlighted the limitations in the controller’s ability to accurately track foot placement on discrete terrains and the lack of consideration for safety concerns. In this study, we address these challenges by focusing on ensuring the overall system’s safety. To begin with, we tackle the inverse kinematics problem by introducing constraints to the damping least squares method. This enhancement not only addresses singularity issues but also guarantees safe ranges for joint angles, thus ensuring the stability and reliability of the system. Based on this, we propose the adoption of the constrained DDPG method to correct controller deviations. In constrained DDPG, we incorporate a constraint layer into the Actor network, incorporating joint deviations as state inputs. By conducting offline training within the range of safe angles, it serves as a deviation corrector. Lastly, we validate the effectiveness of our proposed approach by conducting dynamic simulations using the CRANE biped robot. Through comprehensive assessments, including singularity analysis, constraint effectiveness evaluation, and walking experiments on discrete terrains, we demonstrate the superiority and practicality of our approach in enhancing walking performance while ensuring safety. Overall, our research contributes to the advancement of biped robot locomotion by addressing gait optimisation from multiple perspectives, including singularity handling, safety constraints, and deviation learning.

show abstract

A Deep Deterministic Policy Gradient Approach for Vehicle Speed Tracking Control With a Robotic Driver

Cited by 21 publications

References 20 publications

Deep Deterministic Policy Gradient Algorithm: A Systematic Review

Deep Deterministic Policy Gradient Algorithm: A Systematic Review

Viewpoint planning with transition management for active object recognition

Deep deterministic policy gradient with constraints for gait optimisation of biped robots

Contact Info

Product

Resources

About