Real Time Embedded System Framework for Autonomous Drone Racing using Deep Learning Techniques

Jung, Sunggoo; Lee, Hanseob; Hwang, Sangyeon; Shim, David Hyunchul

doi:10.2514/6.2018-2138

Cited by 11 publications

(5 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep Learning networks are increasingly proposed as solutions for many different problems, thanks to their ability to learn-which they are also capable of under very complex conditions-and their high rates of achieving correct results. These networks have proven to be capable of carrying out even complex tasks [27][28][29]. However, Deep Learning networks also have some disadvantageous features, such as the high level of complexity of the computational structure, a long and complex training procedure, the need for large amounts of training data to ensure accurate and reliable results, and, above all, a very high computational cost [30][31][32][33].…”

Section: Existing Methodologiesmentioning

confidence: 99%

Pattern Orientation Finder (POF): A Robust, Bio-Inspired Light Algorithm for Pattern Orientation Measurement

Carlini,

Paindavoine

2023

Electronics

View full text Add to dashboard Cite

We present the Pattern Orientation Finder (POF), an innovative, bio-inspired algorithm for measuring the orientation of patterns of parallel elements. The POF was developed to obtain an autonomous navigation system for drones inspecting vegetable cultivations. The main challenge was to obtain an accurate and reliable measurement of orientation despite the high level of noise that characterizes aerial views of vegetable crops. The POF algorithm is computationally light and operable on embedded systems. We assessed the performance of the POF algorithm using images of different cultivation types. The outcomes were examined in light of the accuracy and reliability of the measurement; special attention was paid to the relationship between performance and parameterization. The results show that the POF guarantees excellent performance, even in more challenging conditions. The POF shows high reliability and robustness, even in high-noise contexts. Finally, tests on images from different sectors suggest that the POF has excellent potential for application to other fields as well.

show abstract

Section: Existing Methodologiesmentioning

confidence: 99%

Pattern Orientation Finder (POF): A Robust, Bio-Inspired Light Algorithm for Pattern Orientation Measurement

Carlini,

Paindavoine

2023

Electronics

View full text Add to dashboard Cite

show abstract

“…Therefore, computers with specialised image-processing processors or GPUs are employed to split the operations and not saturate the CPU. For example, various authors [3,27] used embedded computers with GPUs such as the NVIDIA Jetson TX1 and TX2, (NVIDIA Corporation, Santa Clara, CA, USA), to improve browsing performance as the GPU speeds up the inference of deep learning networks. Jung et al [27] used an NVIDIA Jetson TX1, which has a quad-core ARM processor and a 192-core Kepler GPU.…”

Section: Related Workmentioning

confidence: 99%

“…For example, various authors [3,27] used embedded computers with GPUs such as the NVIDIA Jetson TX1 and TX2, (NVIDIA Corporation, Santa Clara, CA, USA), to improve browsing performance as the GPU speeds up the inference of deep learning networks. Jung et al [27] used an NVIDIA Jetson TX1, which has a quad-core ARM processor and a 192-core Kepler GPU. They employed a single shot multibox detector (SSD) network for gating detection and reported that by using input images with VGA resolution at 60 Hz, they obtained detection at a speed of 10 Hz.…”

Section: Related Workmentioning

confidence: 99%

Towards Autonomous Drone Racing without GPU Using an OAK-D Smart Camera

Rojas-Perez

Martínez-Carranza

2021

Sensors

View full text Add to dashboard Cite

Recent advances have shown for the first time that it is possible to beat a human with an autonomous drone in a drone race. However, this solution relies heavily on external sensors, specifically on the use of a motion capture system. Thus, a truly autonomous solution demands performing computationally intensive tasks such as gate detection, drone localisation, and state estimation. To this end, other solutions rely on specialised hardware such as graphics processing units (GPUs) whose onboard hardware versions are not as powerful as those available for desktop and server computers. An alternative is to combine specialised hardware with smart sensors capable of processing specific tasks on the chip, alleviating the need for the onboard processor to perform these computations. Motivated by this, we present the initial results of adapting a novel smart camera, known as the OpenCV AI Kit or OAK-D, as part of a solution for the ADR running entirely on board. This smart camera performs neural inference on the chip that does not use a GPU. It can also perform depth estimation with a stereo rig and run neural network models using images from a 4K colour camera as the input. Additionally, seeking to limit the payload to 200 g, we present a new 3D-printed design of the camera’s back case, reducing the original weight 40%, thus enabling the drone to carry it in tandem with a host onboard computer, the Intel Stick compute, where we run a controller based on gate detection. The latter is performed with a neural model running on an OAK-D at an operation frequency of 40 Hz, enabling the drone to fly at a speed of 2 m/s. We deem these initial results promising toward the development of a truly autonomous solution that will run intensive computational tasks fully on board.

show abstract

“…It is now possible to deploy DNN-based computer vision in real time on small unmanned aircraft systems (sUAS) (Zhu et al 2018;Kumaar et al 2020;Castellano et al 2020a,b). Such AI-enhanced systems could automatically scan large or hazardous areas and provide essential situational awareness (Shihavuddin et al 2019;Küchhold et al 2018;Singh, Patil, and Omkar 2018;Jung et al 2018). In-flight processing capabilities are essential for beyondline-of-sight operations, where intermittent, low-bandwidth wireless connections preclude streaming of high-resolution raw imagery back to a base station.…”

Section: Introductionmentioning

confidence: 99%

ADAPT: An Open-Source sUAS Payload for Real-Time Disaster Prediction and Response with AI

Dávila¹,

VanPelt²,

Lynch³

et al. 2022

Preprint

View full text Add to dashboard Cite

Small unmanned aircraft systems (sUAS) are becoming prominent components of many humanitarian assistance and disaster response (HADR) operations. Pairing sUAS with onboard artificial intelligence (AI) substantially extends their utility in covering larger areas with fewer support personnel. A variety of missions, such as search and rescue, assessing structural damage, and monitoring forest fires, floods, and chemical spills, can be supported simply by deploying the appropriate AI models. However, adoption by resourceconstrained groups, such as local municipalities, regulatory agencies, researchers, and indigenous persons, has been hampered by the lack of a cost-effective, readily-accessible baseline platform that can be easily adapted to their unique missions. To fill this gap, we have developed the fully free and open-source ADAPT multi-mission payload for deploying real-time AI and computer vision onboard a sUAS during local and beyond-line-of-site missions. We have emphasized a modular design with low-cost, readily-available components, open-source software, and thorough documentation (https: //kitware.github.io/adapt/). The system integrates an inertial navigation system, high-resolution color camera, computer, and wireless downlink to process imagery and broadcast georegistered analytics back to a ground station. Our goal is to make it easy for the HADR community to build their own copies of the ADAPT payload and leverage the thousands of hours of non-recurring engineering we have devoted to developing and testing this general-purpose capability. In this paper, we detail the development and testing of the ADAPT payload. We also demonstrate the example mission of real-time, in-flight ice segmentation to monitor river ice state and provide more-timely predictions of catastrophic flooding events. We deploy a novel active learning workflow to annotate river ice imagery, train a real-time deep neural network for ice segmentation, and demonstrate operation during field testing.

show abstract

Real Time Embedded System Framework for Autonomous Drone Racing using Deep Learning Techniques

Cited by 11 publications

References 7 publications

Pattern Orientation Finder (POF): A Robust, Bio-Inspired Light Algorithm for Pattern Orientation Measurement

Pattern Orientation Finder (POF): A Robust, Bio-Inspired Light Algorithm for Pattern Orientation Measurement

Towards Autonomous Drone Racing without GPU Using an OAK-D Smart Camera

ADAPT: An Open-Source sUAS Payload for Real-Time Disaster Prediction and Response with AI

Contact Info

Product

Resources

About