Autonomous Drone Racing with Deep Reinforcement Learning

Song, Yunlong; Steinweg, Mats; Kaufmann, Elia; Scaramuzza, Davide

doi:10.1109/iros51168.2021.9636053

Cited by 136 publications

(63 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…A more complex environment for testing algorithms is drone racing. Incorporating deep reinforcement learning and the relative gate observations technique to fly through complex gates has been investigated and appears promising for small-sized drones that can fly up to 60 km/h [14]. Although recent research has used artificial intelligence (AI) for drone control and state estimations, navigation-based techniques are at the center of most works.…”

Section: Literature Reviewmentioning

confidence: 99%

Unstable Landing Platform Pose Estimation Based on Camera and Range Sensor Homogeneous Fusion (CRHF)

Sefidgar

Landry

2022

Drones

View full text Add to dashboard Cite

Much research has been accomplished in the area of drone landing and specifically pose estimation. While some of these works focus on sensor fusion using GPS, or GNSS, we propose a method that uses sensors, including four Time of Flight (ToF) range sensors and a monocular camera. However, when the descending platform is unstable, for example, on ships in the ocean, the uncertainty will grow, and the tracking will fail easily. We designed an algorithm that includes four ToF sensors for calibration and one for pose estimation. The landing process was divided into two main parts, the rendezvous and the final landing. Two important assumptions were made for these two phases. During the rendezvous, the landing platform movement can be ignored, while during the landing phase, the drone is assumed to be stable and waiting for the best time to land. The current research modifies the landing part as a stable drone and an unstable landing platform, which is a Stewart platform, with a mounted AprilTag. A novel algorithm for calibration was used based on color thresholding, a convex hull, and centroid extraction. Next, using the homogeneous coordinate equations of the sensors’ touching points, the focal length in the X and Y directions can be calculated. In addition, knowing the plane equation allows the Z coordinates of the landmark points to be projected. The homogeneous coordinate equation was then used to obtain the landmark’s X and Y Cartesian coordinates. Finally, 3D rigid body transformation is engaged to project the landing platform transformation in the camera frame. The test bench used Software-in-the-Loop (SIL) to confirm the practicality of the method. The results of this work are promising for unstable landing platform pose estimation and offer a significant improvement over the single-camera pose estimation AprilTag detection algorithms (ATDA).

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Unstable Landing Platform Pose Estimation Based on Camera and Range Sensor Homogeneous Fusion (CRHF)

Sefidgar

Landry

2022

Drones

View full text Add to dashboard Cite

show abstract

“…Single-Rotor Thrusts. There are several works that propose to directly learn to control individual rotor thrusts [1], [5], [6], [19]- [22]. As this control input does not require any additional control loop, it provides direct access to the actuators and as a result correctly represents the true actuation limits of the platform.…”

Section: Related Workmentioning

confidence: 99%

“…While [5] required a PID controller at data collection time to facilitate training, [6] demonstrated training of a stabilizing quadrotor control policy from scratch in simulation and deployment on multiple real platforms. [19] trains a policy for autonomous drone racing. Their approach demonstrates competitive racing performance in simulation, but is not deployed on a real quadrotor.…”

Section: Related Workmentioning

confidence: 99%

A Benchmark Comparison of Learned Control Policies for Agile Quadrotor Flight

Kaufmann¹,

Bauersfeld²,

Scaramuzza³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

“…Reinforcement learning has shown promise in learning policies for UAV flight [20]. Deep RL [21] has been used to learn minimum-time trajectory generation for quadrotors. Similar to [20] (but in a multi-agent setting) we train from scratch using DRL, and apply an end-to-end approach: the policies directly control the motor thrust.…”

Section: Related Workmentioning

confidence: 99%

Decentralized Control of Quadrotor Swarms with End-to-end Deep Reinforcement Learning

Batra¹,

Huang²,

Petrenko³

et al. 2021

Preprint

View full text Add to dashboard Cite

We demonstrate the possibility of learning drone swarm controllers that are zero-shot transferable to real quadrotors via large-scale multi-agent end-to-end reinforcement learning. We train policies parameterized by neural networks that are capable of controlling individual drones in a swarm in a fully decentralized manner. Our policies, trained in simulated environments with realistic quadrotor physics, demonstrate advanced flocking behaviors, perform aggressive maneuvers in tight formations while avoiding collisions with each other, break and re-establish formations to avoid collisions with moving obstacles, and efficiently coordinate in pursuit-evasion tasks. We analyze, in simulation, how different model architectures and parameters of the training regime influence the final performance of neural swarms. We demonstrate the successful deployment of the model learned in simulation to highly resource-constrained physical quadrotors performing station keeping and goal swapping behaviors. Code and video demonstrations are available at the project website https://sites.google.com/view/swarm-rl.

show abstract

Autonomous Drone Racing with Deep Reinforcement Learning

Cited by 136 publications

References 24 publications

Unstable Landing Platform Pose Estimation Based on Camera and Range Sensor Homogeneous Fusion (CRHF)

Unstable Landing Platform Pose Estimation Based on Camera and Range Sensor Homogeneous Fusion (CRHF)

A Benchmark Comparison of Learned Control Policies for Agile Quadrotor Flight

Decentralized Control of Quadrotor Swarms with End-to-end Deep Reinforcement Learning

Contact Info

Product

Resources

About