Distributed second‐order multi‐agent constrained optimization algorithm with time‐varying cost function

This paper presents a new formulation of input‐constrained optimal output synchronization problem and proposes an observer‐based distributed optimal control protocol for discrete‐time heterogeneous multiagent systems with input constraints via model‐free reinforcement learning. First, distributed adaptive observers are designed for all agents to estimate the leader's trajectory without requiring its dynamics knowledge. Subsequently, the optimal control input associated with the optimal value function is derived based on the solution to the tracking Hamilton‐Jacobi‐Bellman equation, which is always difficult to solve analytically. To this end, motivated by reinforcement learning technique, a model‐free Q‐learning policy iteration algorithm is proposed, and the actor‐critic neural network structure is implemented to iteratively find the optimal tracking control input without knowing system dynamics. Moreover, inputs of all agents are constrained in the permitted bounds by inserting a nonquadratic function into the performance function, where input constraints are encoded into the optimization problem. Finally, a numerical simulation example is provided to illustrate the effectiveness of the proposed theoretical results.

Section: Introductionmentioning

confidence: 99%

Input‐constrained optimal output synchronization of heterogeneous multiagent systems via observer‐based model‐free reinforcement learning

Zhang

Jia

2023

“…Output constraints can be primarily considered as partial state constraints, which is different from full state constraints. So far, the methods used to solve constrained control problems mainly include prescribed performance control [7], model predictive control (MPC) [8], barrier Lyapunov function (BLF) [9,10] and optimal control [11,12]. In [13], a prescribed performance function is incorporated into adaptive controller design, while suspension travel constraints are considered as state constraints.…”

Section: Introductionmentioning

confidence: 99%

Neural‐based online finite‐time optimal tracking control for wheeled mobile robotic system with inequality constraints

Ding,

Zheng,

et al. 2023

In this study, a finite‐time online optimal controller was designed for a nonlinear wheeled mobile robotic system (WMRS) with inequality constraints, based on reinforcement learning (RL) neural networks. In addition, an extended cost function, obtained by introducing a penalty function to the original long‐time cost function, was proposed to deal with the optimal control problem of the system with inequality constraints. A novel Hamilton‐Jacobi‐Bellman (HJB) equation containing the constraint conditions was defined to determine the optimal control input. Furthermore, two neural networks (NNs), a critic and an actor NN, were established to approximate the extended cost function and the optimal control input, respectively. The adaptation laws of the critic and actor NN were obtained with the gradient descent method. The semi‐global practical finite‐time stability (SGPFS) was proved using Lyapunov's stability theory. The tracking error converges to a small region near zero within the constraints in a finite period. Finally, the effectiveness of the proposed optimal controller was verified by a simulation based on a practical wheeled mobile robot model.

“…In many practical cooperative tasks, nevertheless, the distributed optimization assignments are implemented on continuous-time multi-agent systems, for instance, multi-mobile robots and multi-manipulators. Therefore, continuous-time distributed optimization problem has been considered and a lot of distributed optimization algorithms have been proposed in continuous-time domain for first-order, second-order, and higher order dynamic multi-agent systems [3,[18][19][20][21][22][23][24][25][26][27][28][29][30][31][32].…”

Section: Introductionmentioning

confidence: 99%

Output feedback distributed optimization algorithms of second‐order Lipschitz nonlinear multi‐agent systems

Li,

Yuan

2023

This paper introduces output feedback distributed optimization algorithms designed specifically for second‐order nonlinear multi‐agent systems. The agents are allowed to have heterogeneous dynamics, characterized by distinct nonlinearities, as long as they satisfy the Lipschitz continuity condition. For the case with unknown states, nonlinear state observers are designed first for each agent to reconstruct agents' unknown states. It is proven that the agents' unknown states are estimated accurately by the developed state observers. Then, based on the agents' state estimates and the gradient of each agent local cost function, a kind of output feedback distributed optimization algorithms are proposed for the considered multi‐agent systems. Under the proposed distributed optimization algorithms, all the agents' outputs asymptotically approach the minimizer of the global cost function which is the sum of all the local cost functions. By using Lyapunov stability theory, convex analysis, and input‐to‐state stability theory, the asymptotical convergence of the output feedback distributed optimization closed‐loop system is proven. Simulations are conducted to validate the efficacy of the proposed algorithms.