To improve the efficiency of deep reinforcement learning (DRL)-based methods for robotic trajectory planning in the unstructured working environment with obstacles. Different from the traditional sparse reward function, this paper presents two brand-new dense reward functions. First, the azimuth reward function is proposed to accelerate the learning process locally with a more reasonable trajectory by modeling the position and orientation constraints, which can reduce the blindness of exploration dramatically. To further improve the efficiency, a reward function at subtask-level is proposed to provide global guidance for the agent in the DRL. The subtask-level reward function is designed under the assumption that the task can be divided into several subtasks, which reduces the invalid exploration greatly. The extensive experiments show that the proposed reward functions are able to improve the convergence rate by up to three times with the state-of-the-art DRL methods. The percentage increase in convergence means is 2.25%-13.22% and the percentage decreases with respect to standard deviation by 10.8%-74.5%.INDEX TERMS Deep reinforcement learning, robot manipulator, trajectory planning, reward function.
Rechargeable zinc‐based aqueous batteries are a promising candidate for next‐generation energy storage devices because of the high theoretical energy density of Zn as well as the availability, safety, and cheap price of Zn‐based aqueous batteries. A Zn@C core–shell composite is designed to overcome two problems of Zn anodes, that is, the formation of Zn dendrites and the corrosion/loss of Zn anode during the charge–discharge process, influencing the lifetime of Zn‐based aqueous batteries. The prepared Zn@C composite from ZnO@C with electrochemical reduction is robust as the anode for a Zn@C//MnO2 battery, and a 98.8% retention after 400 cycles is obtained at a discharge rate of 5 C. The specific capacitance of Zn@C//MnO2 battery can reach 135 mAh gZn−1 at 3 C, and the energy density of this battery can reach 75.9 Wh Kg−1. The coated carbon layer on Zn/ZnO plays a role in protecting Zn/ZnO from leaching or corrosion by KOH, so the structure of Zn@C is a key factor in improving the cycling lifetime of Zn‐based aqueous batteries.
To improve the efficiency of surgical trajectory segmentation for robot learning in robot-assisted minimally invasive surgery, this paper presents a fast unsupervised method using video and kinematic data, followed by a promoting procedure to address the over-segmentation issue. Unsupervised deep learning network, stacking convolutional auto-encoder, is employed to extract more discriminative features from videos in an effective way. To further improve the accuracy of segmentation, on one hand, wavelet transform is used to filter out the noises existed in the features from video and kinematic data. On the other hand, the segmentation result is promoted by identifying the adjacent segments with no state transition based on the predefined similarity measurements. Extensive experiments on a public dataset JIGSAWS show that our method achieves much higher accuracy of segmentation than state-of-the-art methods in the shorter time.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.