Double Critics and Double Actors Deep Deterministic Policy Gradient for Mobile Robot Navigation Using Adaptive Parameter Space Noise and Parallel Experience Replay

Hu, Wenjie; Zhou, Ye; Woei Ho, Hann; Zhang, Chun

doi:10.1109/access.2024.3499378

IEEE Access

2024

DOI: 10.1109/access.2024.3499378

|View full text |Cite

Double Critics and Double Actors Deep Deterministic Policy Gradient for Mobile Robot Navigation Using Adaptive Parameter Space Noise and Parallel Experience Replay

Wenjie Hu,

Ye Zhou,

Hann Woei Ho

et al.

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2024

Publication Types

Select...

Article1

Relationship

Self Cite0

Independent1

Authors

Journals

Cited by 1 publication

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Autonomous Maneuvering Decision-Making Algorithm for Unmanned Aerial Vehicles Based on Node Clustering and Deep Deterministic Policy Gradient

Jing,

Cong,

Huang

et al. 2024

Aerospace

View full text Add to dashboard Cite

Decision-making for autonomous maneuvering in dynamic, uncertain, and nonlinear environments represents a challenging frontier problem. Deep deterministic policy gradient (DDPG) is an effective method to solve such problems, but it is found that complex strategies require extensive computation and time in the learning process. To address this issue, we propose a node clustering (NC) method, inspired by grid clustering, integrated into the DDPG algorithm for the learning of complex strategies. In the NC method, the node membership degree is defined according to the specific characteristics of the maneuvering decision-making problem, and error handling strategies are designed to reduce the number of transitions in the replay database effectively, ensuring that the most typical transitions are retained. Then, combining NC and DDPG, an autonomous learning and decision-making algorithm of maneuvering is designed. The algorithm flow and the pseudo-code of the algorithm are given. Finally, the NC_DDPG algorithm is applied to a typical short-range air combat maneuvering decision problem for verification. The results show that the NC_DDPG algorithm significantly accelerates the autonomous learning and decision-making process under both balanced and disadvantageous conditions, taking only about 77% of the time required by Vector DDPG. The scale of NC impacts learning speed; the simulation results across five scales indicate that smaller clustering scales significantly increase learning time, despite a high degree of randomness. Compared with Twin Delayed DDPG (TD3), NC_DDPG consumes only 0.58% of the time of traditional TD3. After applying the NC method to TD3, NC_DDPG requires approximately 20–30% of the time of NC_TD3.

show abstract

Autonomous Maneuvering Decision-Making Algorithm for Unmanned Aerial Vehicles Based on Node Clustering and Deep Deterministic Policy Gradient

Jing,

Cong,

Huang

et al. 2024

Aerospace

View full text Add to dashboard Cite

show abstract

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

customersupport@researchsolutions.com

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

Double Critics and Double Actors Deep Deterministic Policy Gradient for Mobile Robot Navigation Using Adaptive Parameter Space Noise and Parallel Experience Replay

Cited by 1 publication

References 41 publications

Autonomous Maneuvering Decision-Making Algorithm for Unmanned Aerial Vehicles Based on Node Clustering and Deep Deterministic Policy Gradient

Autonomous Maneuvering Decision-Making Algorithm for Unmanned Aerial Vehicles Based on Node Clustering and Deep Deterministic Policy Gradient

Contact Info

Product

Resources

About