Relay Hindsight Experience Replay: Self-guided continual reinforcement learning for sequential object manipulation tasks with sparse rewards

Luo, Yongle; Wang, Yuxin; Dong, Kun; Zhang, Qiang; Cheng, Erkang; Sun, Zhiyong; Song, Bo

doi:10.1016/j.neucom.2023.126620

Cited by 10 publications

(2 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Motion planning for robotic arms usually uses either sparse rewards or dense rewards [16]. When using sparse rewards, in order to efficiently utilize the data, the recently Prioritized Experience Replay (PER) and Hindsight Experience Replay (HER) can achieve good results [17,18]. Chen et al [19] used the shortest distance from the manipulator to the obstacle and the Euclidean distance from the end of the manipulator to the target point as the reward function.…”

Section: Related Workmentioning

confidence: 99%

“…The maximum number of running steps set in the environment was 100, and the environment was re-initialized when the robotic arm took an action that reached 100 steps. The success rate refers to the probability that the distance between the end of the manipulator and the moving target in the environment is achieved within the set threshold value in 100 steps when the algorithm is trained for one round, as shown in Equation (18). In Equation (19), the reach counters are the number of times that the distance between the manipulator arm and the target is less than the threshold value in 100 steps.…”

Section: Experiments and Evaluationmentioning

confidence: 99%

See 1 more Smart Citation

Robotic Manipulator in Dynamic Environment with SAC Combing Attention Mechanism and LSTM

Kuang,

Zhou

2024

Electronics

View full text Add to dashboard Cite

The motion planning task of the manipulator in a dynamic environment is relatively complex. This paper uses the improved Soft Actor Critic Algorithm (SAC) with the maximum entropy advantage as the benchmark algorithm to implement the motion planning of the manipulator. In order to solve the problem of insufficient robustness in dynamic environments and difficulty in adapting to environmental changes, it is proposed to combine Euclidean distance and distance difference to improve the accuracy of approaching the target. In addition, in order to solve the problem of non-stability and uncertainty of the input state in the dynamic environment, which leads to the inability to fully express the state information, we propose an attention network fused with Long Short-Term Memory (LSTM) to improve the SAC algorithm. We conducted simulation experiments and present the experimental results. The results prove that the use of fused neural network functions improved the success rate of approaching the target and improved the SAC algorithm at the same time, which improved the convergence speed, success rate, and avoidance capabilities of the algorithm.

show abstract