Omni-swarm: A Decentralized Omnidirectional Visual-Inertial-UWB State Estimation System for Aerial Swarms

Abstract

The decentralized state estimation is one of the most fundamental components for autonomous aerial swarm systems in GPS-denied areas, which still remains a highly challenging research topic. To address this research niche, the Omniswarm, a decentralized omnidirectional visual-inertial-UWB state estimation system for the aerial swarm is proposed in this paper. In order to solve the issues of observability, complicated initialization, insufficient accuracy and lack of global consistency, we introduce an omnidirectional perception system as the frontend of the Omni-swarm, consisting of omnidirectional sensors, which includes stereo fisheye cameras and ultra-wideband (UWB) sensors, and algorithms, which includes fisheye visual inertial odometry (VIO), multi-drone map-based localization and visual object detector. A graph-based optimization and forward propagation working as the back-end of the Omni-swarm to fuse the measurements from the front-end. According to the experiment result, the proposed decentralized state estimation method on the swarm system achieves centimeter-level relative state estimation accuracy while ensuring global consistency. Moreover, supported by the Omni-swarm, inter-drone collision avoidance can be accomplished in a whole decentralized scheme without any external device, demonstrating the potential of Omniswarm to be the foundation of autonomous aerial swarm flights in different scenarios.

Agile Formation Control of Drone Flocking Enhanced with Active Vision-based Relative Localization

“…The dependence on the external infrastructure of this approach restricts the deployment of the system to unknown environments [18]. Also, it is based on a centralized framework, so it cannot meet the requirements in fully distributed control of swarms [11].…”

“…The detection can be achieved by preknown markers attached to agents [6], [7], or by a pretrained CNN-based detection algorithm [8], [9]. Despite the scalability of the vision-based approach [10], estimation accuracy will deteriorate in the non-line-of-sight case due to restricted FOV [11]. To overcome this limitation, attempts have been made to enlarge the FOV of visual detection in the literature.…”

“…To overcome this limitation, attempts have been made to enlarge the FOV of visual detection in the literature. In [11], distortion-free images extracted from the fisheye camera are used to detect other drones. In [10], four cameras are installed to provide omnidirectional visual inputs.…”

“…Visual sensors have been widely adopted to obtain relative positions in previous works [5]- [11]. The scalability and independence on communication make vision an ideal candidate for relative localization of distributed aerial swarms [10].…”

“…To overcome this limitation, it is intuitive to realize omnidirectional vision detection. Attempts in the literature include fisheye camera [6], [11] and camera array [12]. Nevertheless, the fisheye camera requires additional computational resources to rectify distortion and an array of visual sensors makes the system bulky.…”