Decentralized Multi-Agent Control of a Manipulator in Continuous Task Learning

Abstract

Many real-world tasks require multiple agents to work together. When talking about multiple agents in robotics, it is usually referenced to multiple manipulators in collaboration to solve a given task, where each one is controlled by a single agent. However, due to the increasing development of modular and re-configurable robots, it is also important to investigate the possibility of implementing multi-agent controllers that learn how to manage the manipulator’s degrees of freedom (DoF) in separated clusters for the execution of a given application (e.g., being able to face faults or, partially, new kinematics configurations). Within this context, this paper focuses on the decentralization of the robot control action learning and (re)execution considering a generic multi-DoF manipulator. Indeed, the proposed framework employs a multi-agent paradigm and investigates how such a framework impacts the control action learning process. Multiple variations of the multi-agent framework have been proposed and tested in this research, comparing the achieved performance w.r.t. a centralized (i.e., single-agent) control action learning framework, previously proposed by some of the authors. As a case study, a manipulation task (i.e., grasping and lifting) of an unknown object (to the robot controller) has been considered for validation, employing a Franka EMIKA panda robot. The MuJoCo environment has been employed to implement and test the proposed multi-agent framework. The achieved results show that the proposed decentralized approach is capable of accelerating the learning process at the beginning with respect to the single-agent framework while also reducing the computational effort. In fact, when decentralizing the controller, it is shown that the number of variables involved in the action space can be efficiently separated into several groups and several agents. This simplifies the original complex problem into multiple ones, efficiently improving the task learning process.

Curriculum-reinforcement learning on simulation platform of tendon-driven high-degree of freedom underactuated manipulator

“…On the other hand, constructing simulations often benefits researchers who design robots, helping to confirm the robot’s motion and train it. Software such as Gazebo, CoppeliaSim (previously V-Rep), Multi-Joint dynamics with Contact (MuJoCo), and Matlab have been used to simulate a robot in many studies ( Xiao et al, 2017 ; Huang et al, 2020 ; Shahid et al, 2021 ; Rooban et al, 2022 ). However, in contrast to conventional manipulators with direct motor-driven joints, the relationship between tendons and joints change must be closely considered.…”