Visual model‐predictive localization for computationally efficient autonomous racing of a 72‐g drone

Li, Shuo; Horst, E. van der; Duernay, Philipp; Wagter, Christophe De; Croon, Guido C. H. E. de

doi:10.1002/rob.21956

Cited by 40 publications

(29 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Classical proportional controllers were also used in the work [7]. In this case, they were arranged in a cascade scheme.…”

Section: B Vision-based Drone Controlmentioning

confidence: 99%

A simple vision-based navigation and control strategy for autonomous drone racing

Cyba¹,

Szolc²,

Kryjak³

2021

Preprint

View full text Add to dashboard Cite

In this paper, we present a control system that allows a drone to fly autonomously through a series of gates marked with ArUco tags. A simple and low-cost DJI Tello EDU quad-rotor platform was used. Based on the API provided by the manufacturer, we have created a Python application that enables the communication with the drone over WiFi, realises drone positioning based on visual feedback, and generates control. Two control strategies were proposed, compared, and critically analysed. In addition, the accuracy of the positioning method used was measured.<br>The application was evaluated on a laptop computer (about 40 fps) and a Nvidia Jetson TX2 embedded GPU platform (about 25 fps). We provide the developed code on GitHub. <br>

show abstract

“…Classical proportional controllers were also used in the work [7]. In this case, they were arranged in a cascade scheme.…”

Section: B Vision-based Drone Controlmentioning

confidence: 99%

A simple vision-based navigation and control strategy for autonomous drone racing

Cyba¹,

Szolc²,

Kryjak³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…As an alternative to Kalman filtering, Li et al [33] propose a novel sensor fusion method called Visual Model-Predictive Localization (VML) that incorporates compensation of the delay in the visual information. Experimental results obtained with the VML show a good ability to deal with outliers, an advantage over a simple Kalman filtering.…”

Section: Perception Localization and Filteringmentioning

confidence: 99%

“…Besides that, specific autonomous navigation strategies for a racing drone are also discussed in the literature. Recent papers [24,36], and some previously commented [5,12,33] address methods of localization, control and planning for autonomous navigation in racing environments without obstacles.…”

Section: Solutions For Drone Racingmentioning

confidence: 99%

An integrated solution for an autonomous drone racing in indoor environments

Rezende

Miranda

Rezeck

et al. 2021

Intel Serv Robotics

View full text Add to dashboard Cite

The use of drones is becoming more present in modern daily life. One of the most challenging tasks associated with these vehicles is the development of perception and autonomous navigation systems. Competitions such as Artificial Intelligence Robotic Racing (AIRR) and Autonomous Drone Racing were launched to drive the advances in such strategies, requiring the development of integrated systems for autonomous navigation in a racing environment. In this context, this paper presents an improved integrated solution for autonomous drone racing, which focuses on simple, robust, and computationally efficient techniques to enable the application in embedded hardware. The strategy is divided into four modules: (i) A trajectory planner computes a path that passes through the sequence of desired gates; (ii) a perception system that obtains the global pose of the vehicle by using an onboard camera; (iii) a localization system which merges several sensed information to estimate the drone's states; and, (iv) an artificial vector field-based controller in charge of following the plan by using the estimated states. To evaluate the performance of the entire integrated system, we extended the FlightGoggles simulator to use gates similar to those used in the AIRR competition. A computational cost analysis demonstrates the high performance of the proposed perception method running on limited hardware commonly embedded in aerial robots. In addition, we evaluate the localization system by comparing it with ground truth and when there is a disturbance in the location of the gates. Results in a representative race circuit based on the AIRR competition showed the proposed strategy's competing performance, presenting itself as a feasible solution for drone racing systems.

show abstract

“…The runner-up team used a depth camera on board the drone to generate a local point cloud; the latter was aligned against a 3D CAD model of the race track, thus resolving the drone's localization; waypoints where located some distance before and after the gate for the drone to navigate towards them and to cross them; it must be said that this team achieved the fastest flight in terms of meters per second [1]. Opposite to the use of SLAM, one of the teams proposed a state machine where each state represented a flight behavior [15,16]. The behaviors were defined for different sections in the racetrack.…”

Section: Related Workmentioning

confidence: 99%

DeepPilot: A CNN for Autonomous Drone Racing

Rojas-Perez

Martínez-Carranza

2020

Sensors

View full text Add to dashboard Cite

Autonomous Drone Racing (ADR) was first proposed in IROS 2016. It called for the development of an autonomous drone capable of beating a human in a drone race. After almost five years, several teams have proposed different solutions with a common pipeline: gate detection; drone localization; and stable flight control. Recently, Deep Learning (DL) has been used for gate detection and localization of the drone regarding the gate. However, recent competitions such as the Game of Drones, held at NeurIPS 2019, called for solutions where DL played a more significant role. Motivated by the latter, in this work, we propose a CNN approach called DeepPilot that takes camera images as input and predicts flight commands as output. These flight commands represent: the angular position of the drone’s body frame in the roll and pitch angles, thus producing translation motion in those angles; rotational speed in the yaw angle; and vertical speed referred as altitude h. Values for these 4 flight commands, predicted by DeepPilot, are passed to the drone’s inner controller, thus enabling the drone to navigate autonomously through the gates in the racetrack. For this, we assume that the next gate becomes visible immediately after the current gate has been crossed. We present evaluations in simulated racetrack environments where DeepPilot is run several times successfully to prove repeatability. In average, DeepPilot runs at 25 frames per second (fps). We also present a thorough evaluation of what we called a temporal approach, which consists of creating a mosaic image, with consecutive camera frames, that is passed as input to the DeepPilot. We argue that this helps to learn the drone’s motion trend regarding the gate, thus acting as a local memory that leverages the prediction of the flight commands. Our results indicate that this purely DL-based artificial pilot is feasible to be used for the ADR challenge.

show abstract

Visual model‐predictive localization for computationally efficient autonomous racing of a 72‐g drone

Cited by 40 publications

References 31 publications

A simple vision-based navigation and control strategy for autonomous drone racing

A simple vision-based navigation and control strategy for autonomous drone racing

An integrated solution for an autonomous drone racing in indoor environments

DeepPilot: A CNN for Autonomous Drone Racing

Contact Info

Product

Resources

About