Formation control scheme with reinforcement learning strategy for a group of multiple surface vehicles

Abstract

This article presents a comprehensive approach to integrate formation tracking control and optimal control for a fleet of multiple surface vehicles (SVs), accounting for both kinematic and dynamic models of each SV agent. The proposed control framework comprises two core components: a high‐level displacement‐based formation controller and a low‐level reinforcement learning (RL)‐based optimal control strategy for individual SV agents. The high‐level formation control law, employing a modified gradient method, is introduced to guide the SVs in achieving desired formations. Meanwhile, the low‐level control structure, featuring time‐varying references, incorporates the RL algorithm by transforming the time‐varying closed agent system into an equivalent autonomous system. The application of Lyapunov's direct approach, along with the existence of the Bellman function, guarantees the stability and optimality of the proposed design. Through extensive numerical simulations, encompassing various comparisons and scenarios, this study demonstrates the efficacy of the novel formation control strategy for multiple SV agent systems, showcasing its potential for real‐world applications.

Bridging Requirements, Planning, and Evaluation: A Review of Social Robot Navigation

“…Optimal control techniques have also been employed to maintain the formation integrity [330,331]. For instance, in [330], formation control in a leader-follower arrangement was discussed.…”

An Integrated Autonomous Dynamic Navigation Approach toward a Composite Air–Ground Risk Construction Scenario

“…Differing from typical hierarchical decision approaches, which are based on temporal or informational flow [ 20 , 21 ], the proposed non-steering/steering-based hierarchical approach in this paper aims to decrease the destabilizing risks associated with rapid steering actions. Non-steering avoidance involves trajectory adjustments through pure acceleration or deceleration without altering the vehicle’s original path, primarily focusing on longitudinal obstacle avoidance.…”

“…In Table A1, v o x (0) and v o y (0) denote the initial horizontal and longitudinal velocity of the obstacles, respectively. Once the motion of obstacles extends beyond the rectangular boundary that originated from the coordinates (20,20) to (80, 80), the subsequent speed will be set to:…”

Towards Reliable Federated Learning Using Blockchain-Based Reverse Auctions and Reputation Incentives

“…The local model of a node is accepted by BCD-FL only when m t i ≥ δ, otherwise it is discarded. The federated average aggregation algorithm generally follows Equation (16).…”

“…Other research initiatives, such as those detailed in [13,14], have used Stackelberg game theory and Deep Reinforcement Learning (DRL) to encourage participant engagement. This success can be attributed to the fact that reinforcement learning [15,16] has been proven to be a highly effective method in various fields of artificial intelligence. On the other hand, these solutions do not take into account the influence that the quality of model updates might have on the effectiveness of the learning process.…”