Constrained Motion Planning of 7-DOF Space Manipulator via Deep Reinforcement Learning Combined with Artificial Potential Field

Li, Yinkang; Li, Danyi; Zhu, Wenshan; Sun, Jun; Zhang, Xiaolong; Li, Shuang

doi:10.3390/aerospace9030163

Cited by 23 publications

(17 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Reinforcement learning is an overall process that refers to the agent’s trial, evaluation, and action memory ( Clifton and Laber, 2020 ; Chen et al, 2022 ; Cong et al, 2022 ; Li et al, 2022 ). The agent’s learning maps from environment state to action, causing it to reap the greatest rewards after carrying out a particular action.…”

Section: Methodsmentioning

confidence: 99%

Reinforcement learning based variable damping control of wearable robotic limbs for maintaining astronaut pose during extravehicular activity

Zhao

Zheng²,

Sui³

et al. 2023

Front. Neurorobot.

View full text Add to dashboard Cite

As astronauts perform on-orbit servicing of extravehicular activity (EVA) without the help of the space station’s robotic arms, it will be rather difficult and labor-consuming to maintain the appropriate position in case of impact. In order to solve this problem, we propose the development of a wearable robotic limb system for astronaut assistance and a variable damping control method for maintaining the astronaut’s position. The requirements of the astronaut’s impact-resisting ability during EVA were analyzed, including the capabilities of deviation resistance, fast return, oscillation resistance, and accurate return. To meet these needs, the system of the astronaut with robotic limbs was modeled and simplified. In combination with this simplified model and a reinforcement learning algorithm, a variable damping controller for the end of the robotic limb was obtained, which can regulate the dynamic performance of the robot end to resist oscillation after impact. A weightless simulation environment for the astronaut with robotic limbs was constructed. The simulation results demonstrate that the proposed method can meet the recommended requirements for maintaining an astronaut’s position during EVA. No matter how the damping coefficient was set, the fixed damping control method failed to meet all four requirements at the same time. In comparison to the fixed damping control method, the variable damping controller proposed in this paper fully satisfied all the impact-resisting requirements by itself. It could prevent excessive deviation from the original position and was able to achieve a fast return to the starting point. The maximum deviation displacement was reduced by 39.3% and the recovery time was cut by 17.7%. Besides, it also had the ability to prevent reciprocating oscillation and return to the original position accurately.

show abstract

Section: Methodsmentioning

confidence: 99%

Reinforcement learning based variable damping control of wearable robotic limbs for maintaining astronaut pose during extravehicular activity

Zhao

Zheng²,

Sui³

et al. 2023

Front. Neurorobot.

View full text Add to dashboard Cite

show abstract

“…[32] 2017 -Initialization of the Q-table by the APF implementation to update the Q-table to increase convergence speed and quality before training phase. [33] 2022 [34] 2021 -Utilize the real-time information as input to DQL.…”

Section: Refmentioning

confidence: 99%

Online path planning of cooperative mobile robots in unknown environments using improved Q‐Learning and adaptive artificial potential field

Ataollahi

Farrokhi

2023

The Journal of Engineering

View full text Add to dashboard Cite

Online path planning in unknown environments with obstacles is a challenging issue in mobile robot systems. Cooperative mobile robots have received much attention in recent years due to their robustness and ability to perform more complex tasks and operations faster. In this article, by unifying the Dyna Q-learning and a novel concept (introduced as a feature matrix) an improved Q-learning is proposed that accelerates the learning process even with poor choice of parameters. In addition, to overcoming the local minima, an adaptive artificial potential field method is developed, so that two parameters of the new concept named the virtual obstacle are controlled using the massachusetts institute of technology (MIT) method to navigate the mobile robot in the proper direction. Then, with proper utilizing of the improved Q-learning and adaptive artificial potential field method, the path planning is performed online effectively while target tracking and collision avoidance are guaranteed. Finally, the performance of the proposed method is tested on decentralized cooperative mobile robots. The simulation results showed optimal path planning in terms of the distance in an unknown environment and collision avoidance with the obstacles. In addition, getting out of the local minima (i.e. immobility) is guaranteed in all situations.

show abstract

“…Therefore, the proposed approach can be applied to various space operation tasks. Subsequently, they combined deep reinforcement learning algorithms with artificial potential fields to improve convergence and robustness [105]. Yu [106] used the DDPG framework to learn a redundant robotic arm dual-arm path planning strategy to achieve the capture of target satellites by a spatial spherical-revolute-spherical (SRS) redundant robotic arm.…”

Section: Application Of Reinforcement Learning Algorithmsmentioning

confidence: 99%

A Review of Spatial Robotic Arm Trajectory Planning

et al. 2022

View full text Add to dashboard Cite

With space technology development, the spatial robotic arm plays an increasingly important role in space activities. Spatial robotic arms can effectively replace humans to complete in-orbit service tasks. The trajectory planning is the basis of robotic arm motion. Its merit has an essential impact on the quality of the completed operation. The research on spatial robotic arm trajectory planning has not yet formed a broad framework categorization, so it is necessary to analyze and deeply summarize the existing research systematically. This paper introduces the current situation of space obstacle avoidance trajectory planning and motion trajectory planning. It discusses the basic principle and practical application of the spatial robotic arm trajectory planning method. The future development trend has also been prospected.

show abstract

Constrained Motion Planning of 7-DOF Space Manipulator via Deep Reinforcement Learning Combined with Artificial Potential Field

Cited by 23 publications

References 35 publications

Reinforcement learning based variable damping control of wearable robotic limbs for maintaining astronaut pose during extravehicular activity

Reinforcement learning based variable damping control of wearable robotic limbs for maintaining astronaut pose during extravehicular activity

Online path planning of cooperative mobile robots in unknown environments using improved Q‐Learning and adaptive artificial potential field

A Review of Spatial Robotic Arm Trajectory Planning

Contact Info

Product

Resources

About