Cooperative Object Transportation Using Curriculum-Based Deep Reinforcement Learning

Abstract

This paper presents a cooperative object transportation technique using deep reinforcement learning (DRL) based on curricula. Previous studies on object transportation highly depended on complex and intractable controls, such as grasping, pushing, and caging. Recently, DRL-based object transportation techniques have been proposed, which showed improved performance without precise controller design. However, DRL-based techniques not only take a long time to learn their policies but also sometimes fail to learn. It is difficult to learn the policy of DRL by random actions only. Therefore, we propose two curricula for the efficient learning of object transportation: region-growing and single- to multi-robot. During the learning process, the region-growing curriculum gradually extended to a region in which an object was initialized. This step-by-step learning raised the success probability of object transportation by restricting the working area. Multiple robots could easily learn a new policy by exploiting the pre-trained policy of a single robot. This single- to multi-robot curriculum can help robots to learn a transporting method with trial and error. Simulation results are presented to verify the proposed techniques.

Multi-Agent Deep Reinforcement Learning for Multi-Robot Applications: A Survey

“…Ground Robots Manipulators [23,24,38,46,53,56,57,68,76,83,85,91,104,106,110,110,120,123,134,135,138,139,141,142,161,167,171,172,186,205,207,[218][219][220]230,240,269] [ 22,39,49,52,54,55,59,70,71,[73][74][75][77][78][79]…”

“…Manko et al [ 77 ] used CNN-based DRL architecture for multi-robot collaborative transportation where the objective is to carry an object from the start to the goal location. Eoh and Park [ 79 ] proposed a curriculum-based deep reinforcement learning method for training robots to cooperatively transport an object. In a curriculum-based RL, past experiences are organized and sorted to improve training efficiency [ 252 ].…”

Guided Reinforcement Learning: A Review and Evaluation for Efficient and Effective Real-World Robotics [Survey]

“…For instance, Luo et al [78] exploit a curriculum that gradually adjusts the precision requirements for multigoal reach experiments and show that it improves performance in a faster way. Eoh et al [29], on the other hand, employ a curriculum learning approach for challenging multirobot object transportation tasks that gradually increases both the transportation distance and number of robots involved. Leyendecker et al [70] propose a combination of reward curriculum and domain randomization to develop a robust sim-to-real transferable policy to execute a manipulation task in an industrial setup.…”

Estimation and Control of Cooperative Aerial Manipulators for a Payload with an Arbitrary Center-of-Mass

“…(2) At each grasping point, the roll and pitch angle of the end effector is the same for the cooperative aerial manipulators. (3) Configurations of the payload are previously given, so aerial manipulators know the grasping point of the payload. However, the exact relative distances and heading angles between each robot are unknown.…”

“…Among them, cooperative UAVs are widely exploited to handle a heavy or large payload [ 2 ] beyond the limits of a robot’s transportation capabilities. Recently, researchers have developed cooperative mobile manipulators [ 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 ] by exploiting grasping capability. However, due to several issues including the complexity associated with multiple aerial robots, they have focused on solving a control and coordination problem.…”