Autonomous aerial robot using dual‐fisheye cameras

Gao, Wenliang; Kaixuan, Wang; Ding, Wenchao; Gao, Fei; Qin, Tong; Shen, Shaojie

doi:10.1002/rob.21946

Cited by 30 publications

(15 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Based on previous works on fisheye VIO [3], [26], [27], we developed a fisheye visual-inertial navigation system to estimate VIO: VINS-Fisheye 1 . Because of the massive distortion, it is hard to directly apply the existing visual algorithms to the raw image data produced by the fisheye camera.…”

Section: A Fisheye Visual Inertial Odometrymentioning

confidence: 99%

“…Because of the massive distortion, it is hard to directly apply the existing visual algorithms to the raw image data produced by the fisheye camera. As an alternative, we use the methods described in the article [26], [27] to crop regions of the original fisheye image and reproject it into five distortion-free images for later algorithms. An example of raw image and processed distortion-free images is shown in Fig.…”

Section: A Fisheye Visual Inertial Odometrymentioning

confidence: 99%

See 1 more Smart Citation

Omni-swarm: A Decentralized Omnidirectional Visual-Inertial-UWB State Estimation System for Aerial Swarms

Xu¹,

Zhang²,

Zhou³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

The decentralized state estimation is one of the most fundamental components for autonomous aerial swarm systems in GPS-denied areas, which still remains a highly challenging research topic. To address this research niche, the Omniswarm, a decentralized omnidirectional visual-inertial-UWB state estimation system for the aerial swarm is proposed in this paper. In order to solve the issues of observability, complicated initialization, insufficient accuracy and lack of global consistency, we introduce an omnidirectional perception system as the frontend of the Omni-swarm, consisting of omnidirectional sensors, which includes stereo fisheye cameras and ultra-wideband (UWB) sensors, and algorithms, which includes fisheye visual inertial odometry (VIO), multi-drone map-based localization and visual object detector. A graph-based optimization and forward propagation working as the back-end of the Omni-swarm to fuse the measurements from the front-end. According to the experiment result, the proposed decentralized state estimation method on the swarm system achieves centimeter-level relative state estimation accuracy while ensuring global consistency. Moreover, supported by the Omni-swarm, inter-drone collision avoidance can be accomplished in a whole decentralized scheme without any external device, demonstrating the potential of Omniswarm to be the foundation of autonomous aerial swarm flights in different scenarios.

show abstract

Section: A Fisheye Visual Inertial Odometrymentioning

confidence: 99%

Section: A Fisheye Visual Inertial Odometrymentioning

confidence: 99%

Omni-swarm: A Decentralized Omnidirectional Visual-Inertial-UWB State Estimation System for Aerial Swarms

Xu¹,

Zhang²,

Zhou³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…An existing issue for UAVs is that the small field of view (FoV) of these sensors severely limits the UAV's perception capability and task efficiency. Although many efforts have been made to deal with the constraints in applications induced by narrow or limited FoVs (1)(2)(3)(4)(5), a larger FoV is still a better solution that not only reduces task time by observing the environment more efficiently (6) but also enhances the UAV safety in the wild by perceiving dynamic obstacles (e.g., birds) approaching from an unknown direction (7).…”

Section: Introductionmentioning

confidence: 99%

A self-rotating, single-actuated UAV with extended sensor field of view for autonomous navigation

et al. 2023

View full text Add to dashboard Cite

Uncrewed aerial vehicles (UAVs) rely heavily on visual sensors to perceive obstacles and explore environments. Current UAVs are limited in both perception capability and task efficiency because of a small sensor field of view (FoV). One solution could be to leverage self-rotation in UAVs to extend the sensor FoV without consuming extra power. This natural mechanism, induced by the counter-torque of the UAV motor, has rarely been exploited by existing autonomous UAVs because of the difficulties in design and control due to highly coupled and nonlinear dynamics and the challenges in navigation brought by the high-rate self-rotation. Here, we present powered-flying ultra-underactuated LiDAR (light detection and ranging) sensing aerial robot (PULSAR), an agile and self-rotating UAV whose three-dimensional position is fully controlled by actuating only one motor to obtain the required thrust and moment. The use of a single actuator effectively reduces the energy loss in powered flights. Consequently, PULSAR consumes 26.7% less power than the benchmarked quadrotor with the same total propeller disk area and avionic payloads while retaining a good level of agility. Augmented by an onboard LiDAR sensor, PULSAR can perform autonomous navigation in unknown environments and detect both static and dynamic obstacles in panoramic views without any external instruments. We report the experiments of PULSAR in environment exploration and multidirectional dynamic obstacle avoidance with the extended FoV via self-rotation, which could lead to increased perception capability, task efficiency, and flight safety.

show abstract

“…Real-time and robust stereo vision systems for the computation of depth information are increasingly popular in many embedded applications including robotic navigation [1,2] and autonomous vehicles [3,4]. Stereo matching methods take a pair of left-right images from stereo cameras as input and generate the disparity map for depth estimation.…”

Section: Introductionmentioning

confidence: 99%

FP-Stereo: Hardware-Efficient Stereo Vision for Embedded Applications

Zhao

Liang

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Fast and accurate depth estimation, or stereo matching, is essential in embedded stereo vision systems, requiring substantial design effort to achieve an appropriate balance among accuracy, speed and hardware cost. To reduce the design effort and achieve the right balance, we propose FP-Stereo for building highperformance stereo matching pipelines on FPGAs automatically. FP-Stereo consists of an open-source hardware-efficient library, allowing designers to obtain the desired implementation instantly. Diverse methods are supported in our library for each stage of the stereo matching pipeline and a series of techniques are developed to exploit the parallelism and reduce the resource overhead. To improve the usability, FP-Stereo can generate synthesizable C code of the FPGA accelerator with our optimized HLS templates automatically. To guide users for the right design choice meeting specific application requirements, detailed comparisons are performed on various configurations of our library to investigate the accuracy/speed/cost trade-off. Experimental results also show that FP-Stereo outperforms the state-of-the-art FPGA design from all aspects, including 6.08% lower error, 2x faster speed, 30% less resource usage and 40% less energy consumption. Compared to GPU designs, FP-Stereo achieves the same accuracy at a competitive speed while consuming much less energy.

show abstract

Autonomous aerial robot using dual‐fisheye cameras

Cited by 30 publications

References 41 publications

Omni-swarm: A Decentralized Omnidirectional Visual-Inertial-UWB State Estimation System for Aerial Swarms

Omni-swarm: A Decentralized Omnidirectional Visual-Inertial-UWB State Estimation System for Aerial Swarms

A self-rotating, single-actuated UAV with extended sensor field of view for autonomous navigation

FP-Stereo: Hardware-Efficient Stereo Vision for Embedded Applications

Contact Info

Product

Resources

About