Learning-Based Multi-UAV Flocking Control With Limited Visual Field and Instinctive Repulsion

Bai, Chengchao; Yan, Peng; Piao, Haiyin; Pan, Wei; Guo, Jifeng

doi:10.1109/tcyb.2023.3246985

Cited by 14 publications

(3 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The analysis of the data, as presented in Table 1 and illustrated in Figure 12, reveals a comparative evaluation of the average and standard deviation of the performance metrics across different methods. A noteworthy observation from the analysis is that our proposed method significantly surpasses the performance metrics of the other three benchmark algorithms (sac [36], ppo [37], and ddpg [38]), highlighting the superior efficacy of our method over alternative strategies. This superior performance is not merely confined to a comparison with benchmark algorithms, but it extends to a close alignment with the metrics directed by the expert policy, underscoring the potent capability of imitation learning within this framework.…”

Section: Performance Analysismentioning

confidence: 84%

“…In order to compare the performance difference between our method and other benchmark algorithms, we define the performance metrics and conduct a relevant analysis. The benchmark algorithms are those such as sac [36], ppo [37], and ddpg [38]. We assessed the performance of the five policies in the pre-established scenario, ensuring that the environmental settings remained consistent with those during training.…”

Section: Comparative Analysismentioning

confidence: 99%

“…End-to-end strategies for multi-UAV systems often leverage machine learning or optimization techniques, including reinforcement learning methods such as sac [36], ppo [37], ddpg [38], etc., as well as imitation learning methods [9,[33][34][35]. Among these, imitation learning holds particular promise for autonomous flight due to its unique advantages.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Imitation Learning of Complex Behaviors for Multiple Drones with Limited Vision

Wan,

Tang,

Zhao

2023

Drones

View full text Add to dashboard Cite

Navigating multiple drones autonomously in complex and unpredictable environments, such as forests, poses a significant challenge typically addressed by wireless communication for coordination. However, this approach falls short in situations with limited central control or blocked communications. Addressing this gap, our paper explores the learning of complex behaviors by multiple drones with limited vision. Drones in a swarm rely on onboard sensors, primarily forward-facing stereo cameras, for environmental perception and neighbor detection. They learn complex maneuvers through the imitation of a privileged expert system, which involves finding the optimal set of neural network parameters to enable the most effective mapping from sensory perception to control commands. The training process adopts the Dagger algorithm, employing the framework of centralized training with decentralized execution. Using this technique, drones rapidly learn complex behaviors, such as avoiding obstacles, coordinating movements, and navigating to specified targets, all in the absence of wireless communication. This paper details the construction of a distributed multi-UAV cooperative motion model under limited vision, emphasizing the autonomy of each drone in achieving coordinated flight and obstacle avoidance. Our methodological approach and experimental results validate the effectiveness of the proposed vision-based end-to-end controller, paving the way for more sophisticated applications of multi-UAV systems in intricate, real-world scenarios.

show abstract

Section: Performance Analysismentioning

confidence: 84%

Section: Comparative Analysismentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Imitation Learning of Complex Behaviors for Multiple Drones with Limited Vision

Wan,

Tang,

Zhao

2023

Drones

View full text Add to dashboard Cite

show abstract

Digital Twin‐Enabled Deep Reinforcement Learning for Safety‐Guaranteed Flocking Motion of UAV Swarm

Li,

Lei,

Shen

et al. 2024

Trans Emerging Tel Tech

View full text Add to dashboard Cite

Multi‐agent deep reinforcement learning (MADRL) has become a typical paradigm for the flocking motion of UAV swarm in dynamic, stochastic environments. However, sim‐to‐real problems, such as reality gap, training efficiency, and safety issues, restrict the application of MADRL in flocking motion scenarios. To address these problems, we first propose a digital twin (DT)‐enabled training framework. With the assistance of high‐fidelity digital twin simulation, effective policies can be efficiently trained. Based on the multi‐agent proximal policy optimization (MAPPO) algorithm, we then design the learning approach for flocking motion with matching observation space, action space, and reward function. Afterward, we employ a distributed flocking center estimation algorithm based on position consensus. The estimated center is used as a policy input to improve the aggregation behavior. Moreover, we introduce a repulsion scheme, which applies an additional repulsion force to the action to prevent UAVs from colliding with neighbors and obstacles. Simulation results show that our method performs well in maintaining flocking formation and avoiding collisions, and has better decision‐making ability in near‐realistic environments.

show abstract