Data-Driven Distributed Optimal Consensus Control for Unknown Multiagent Systems With Input-Delay

Abstract

This paper is concerned with data-driven distributed optimal consensus control for unknown multiagent systems (MASs) with input delays. The input-delayed MAS model is first converted into a delay-free form using a model reduction method. By establishing an equivalent relationship on the predesigned performance indices of the two MASs, optimal consensus control of input-delayed MAS can be fully transformed to that of delay-free MAS. Based on the coupled Hamilton-Jacobi equations and Bellman's optimality principle, optimal consensus control policies are derived for the transformed delay-free MAS. Then a policy iteration algorithm based on distributed asynchronous update mechanism is proposed to learn the coupled Hamilton-Jacobi-Bellman equations online. To perform the proposed data-driven adaptive dynamic programming algorithm, we adopt the measured data-based critic-actor neural networks to approximate the value functions and the control policies, respectively. Finally, a simulation example is given to illustrate the effectiveness of the proposed method.

Distributed Model-Free Bipartite Consensus Tracking for Unknown Heterogeneous Multi-Agent Systems with Switching Topology

“… Both collaborative and antagonistic interactions among agents are considered in the proposed protocol. Compared with the protocols in [ 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 ], the proposed protocol is more reasonable. Moreover, the difference of DMFABCT from the novel algorithm proposed in [ 32 ] is that DMFABCT copes with the BC problem with PPD, where the training processes and external testing signals are not necessary.…”

“…Recently, several papers [12][13][14][15][16][17][18][19][20][21][22][23][24] have reported on model-free adaptive control (MFAC), interactive learning control (ILC), repetitive learning control (RLC), reinforcement learning (RL), and so on. The consensus tracking problems of MASs were researched in [12] by the MFAC approach, where both the time invariable and varying desired trajectories tracking are archived.…”

“…In our investigations, we found that another category of MFC methods is based on neural networks (NNs), which have unparalleled approximation abilities for nonlinear dynamics. In [16][17][18], the authors designed actor-critic-based neural networks to approximate the value function and control policy for each agent, respectively, to optimize consensus control performance. It should be pointed out that NNs-based methods need training processes and external testing signals for controller design, which are not convenient.…”

“…In the aforementioned related studies [ 5 , 6 , 7 , 8 , 9 , 10 , 11 ], consensus problems of MASs are based on MBC approaches, while the authors of [ 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 ] employed and developed MFC methods to address consensus or consensus tracking problems for MASs; however, it is still an open and challenging problem for unknown dynamics MASs to achieve consensus tracking. Furthermore, it is obvious from a review of the above literature that MASs consensus control and tracking only consider the cooperation interactions among agents.…”

Robust Optimal Formation Control of Heterogeneous Multi-Agent System via Reinforcement Learning

“…In [23], a model-free RL method was applied to achieve the optimal consensus for networked systems without knowledge of the agent dynamic model. In [24], the RL approaches were used to achieve the data-driven optimal consensus control for MASs subject to input-delay. However, the works in [19]- [24] mainly focused on the optimal consensus of MASs and the robust control of heterogeneous MASs subject to external disturbances and parameter uncertainties was not investigated.…”