Robot grasping method optimization using improved deep deterministic policy gradient algorithm of deep reinforcement learning

Abstract

Robot grasping has become a very hot research field so that the requirements for robot operation are getting higher and higher. In previous research studies, the use of traditional target detection algorithms for grasping is often very inefficient, and this article is dedicated to improving the deep reinforcement learning algorithm to improve the grasping efficiency and solve the problem of robots dealing with the impact of unknown disturbances on grasping. Using the characteristic that deep reinforcement learning actively explores the unknown environment, a Gaussian parameter Deep Deterministic Policy Gradient (Gaussian-DDPG) algorithm based on the Importance-Weighted Autoencoder (IWAE) is proposed to realize the robot’s autonomous learning of the grasping task. Traditional coordinate positioning methods and deep learning methods have poor grasping effects for disturbed situations (such as the movement of the target object). The IWAE algorithm is used to compress the high-dimensional information of the original visual input to the hidden space and pass it to the deep reinforcement learning network as part of the state value. Based on the classic DDPG algorithm, it smoothly adds Gaussian parameters to improve the exploratory nature of the algorithm, dynamically sets the robot grasping space parameters to adapt to the workspace of multiple scales, and finally, realizes the accurate grasping of the robot. Relying on the possible position information deviation of the visual information, the control of the grasping position by the manipulator torque information is further optimized to improve the grasping efficiency of disturbed objects.

Advancements in Deep Reinforcement Learning and Inverse Reinforcement Learning for Robotic Manipulation: Toward Trustworthy, Interpretable, and Explainable Artificial Intelligence

“…b) Policy-based methods are algorithms that directly optimize policy without estimating the value of states or state-action pairs by determining the actions taken by the agent as a function of the agent's state and environment [27]. Policy Gradient [44], Advantage Actor-Critic (A2C) [45], Asynchronous Advantage Actor-Critic (A3C) [46], Proximal Policy Optimization (PPO) [47] In robotic manipulation the classification of DRL is given according to the specific tasks and problem-solving approaches: Grasping and manipulation: Applications include using DRL for robots to perform ambiguous manipulation tasks such as grasping and hand manipulation [12]. Navigation and localization: Applications where DRL is not used for robots to navigate and localize different environments Figure 2 shows some examples from the studies on manipulation tasks such as robotic grasping [57], robotic hand manipulation [58], and object manipulation [59] using DRL algorithms.…”

Deep Deterministic Policy Gradient Algorithm: A Systematic Review

“…DDPG algorithm has been successfully applied to Robotics and motion control problems [81,82]. DDPG can be used for a variety of tasks, such as manipulation, locomotion, and navigation [83].…”

“…Robotic manipulation (H. X. Zhang et al, 2021) [81] Techniques: Importance-Weighted Autoencoder (IWAE) and Gaussian parameter (Gaussian-DDPG) Methodology: Addition of Gaussian parameters to DDPG algorithm for better exploration and optimization of grasping position control using torque information.…”

Goal-Conditioned Reinforcement Learning within a Human-Robot Disassembly Environment

“…The effective resolution of complex manipulation tasks in an unstructured or highly variable environment remains an active field of research. Current research focuses mainly on grasping [12], picking and placing [13], and assembly tasks [14]. In particular, RL methods have shown high robustness to uncertainties in the latter, leading more and more researchers to focus on learning assembly skills.…”