Robust Optimal Formation Control of Heterogeneous Multi-Agent System via Reinforcement Learning

Abstract

In this paper, a distributed robust optimal formation control problem is studied based on reinforcement learning for the heterogeneous multi-agent system with partial unknown system parameters. The formation system is subjected to equivalent disturbances containing parameter uncertainties and external disturbances. The proposed robust optimal controller consists of a nominal controller and a robust compensator. For the nominal controller, the reinforcement learning algorithm is proposed to obtain the optimal control input. For the robust compensator, the reinforcement learning algorithm is firstly used to identify the unknown dynamic parameters and then the robust compensator is designed to restrain the equivalent disturbances in the formation system. The robustness properties of the global multi-agent system are proven. A simulation of heterogeneous rotorcrafts is provided to verify the effectiveness of the proposed method. INDEX TERMS Formation control, reinforcement learning, robust optimal control, heterogeneous multiagent systems.

Heterogeneous swarm control based on two‐layer topology

“…In heterogeneous swarm control, every agent may have different dynamic parameters and functions 12,13 . In such cases, it is difficult to apply the control method of homogeneous MAS to accomplish the formation mission of heterogeneous MAS 14 . Authors 15 simultaneously consider three problems in heterogeneous MAS and design an adaptive neuro‐fuzzy controller to achieve synchronization between followers and a leader.…”

Distributed Observer-Based Leader-Following Consensus Control Robust to External Disturbance and Measurement Sensor Noise for LTI Multi-agent Systems

“…In [7], a robust controller using 𝐻 2 performance is implemented in a system with communication and input time delays presented in the frequency domain minimizing the error and the disturbance effect. In [8], the robust optimal formation control problem for heterogeneous multi-agent systems considering external disturbances is addressed based on reinforcement learning. An adaptive semi-global bipartite consensus assuming a connected switching topology graph under input saturation and external disturbance is proposed in [9].…”

HDP Algorithms for Trajectory Tracking and Formation Control of Multi-Agent Systems

“…This function evaluates the effect of control on the future performance and provides guidelines on how to improve the control law. These structures are combined with reinforcement learning (RL), which has been used as a framework for solving real-time optimal decision problems [18], [19].…”