Heterogeneous Parallelization for Object Detection and Tracking in UAVs

Abstract

Recent technical advancements in both fields of unmanned aerial vehicles (UAV) control and artificial intelligence (AI) have made a certain realm of applications possible. However, one of the main problems in integration of these two areas is the bottleneck of computing AI applications on UAV's resource limited platform. One of the main solution for this problem is that AI and control software from one side and computing hardware mounted on UAV from the other side be adopted together based on the main constraints of the resource limited computing platform on UAV. Basically, the target constraints of such adaptation are performance, energy efficiency, and accuracy. In this paper, we propose a strategy to integrate and adopt the commonly used object detection and tracking algorithm and UAV control software to be executed on a heterogeneous resource limited computing units on a UAV. For object detection, a convolutional neural network (CNN) algorithm is used. For object tracking, a novel algorithm is proposed that can execute along with object tracking via sequential stream data. For UAV control, a Gain-Scheduled PID controller is designed that steers the UAV by continuously manipulation of the actuators based on the stream data from the tracking unit and dynamics of the UAV. All the algorithms are adopted to be executed on a heterogeneous platform including NVIDIA Jetson TX2 embedded computer and an ARM Cortex M4. The observation from real-time operation of the platform shows that using the proposed platform reduces the power consumption by 53.69% in contrast with other existing methods while having marginal penalty for object detection and tracking parts.

An AI-in-Loop Fuzzy-Control Technique for UAV’s Stabilization and Landing

“…In [16], the authors proposed a control system for stabilizing the UAV that is following a moving object under various speeds. To do this, a Gain-Scheduled PID controller (GPID) has been adopted that refines the system's actuation based on the received feedback.…”

“…To show the efficiency of the proposed technique with the state-of-the-art, we implemented different control algorithms for both stabilization and landing parts. For the landing part we implemented the pure PID control algorithm for UAVs in [24] and [25], called as L-PID in this paper, and for the stabilization algorithm we implemented a Gain-scheduled PID proposed in [16] and a fuzzy control proposed in [20] (called S-GPID and S-FLC respectively). Our proposed control algorithm in the comparison is shown by S-ours and L-ours for stabilization and landing respectively.…”

Detection algorithm of safety helmet wearing based on deep learning

“…Object detection is one of the three basic tasks in the field of computer vision, 4 which is parallel to the other two basic tasks of image processing: image classification 5 and image semantic segmentation 6‐8 . Object detection refers to finding the target object in the input image.…”

Vision-Based Target Detection and Tracking System for a Quadcopter

“…In [27], Justin et al addressed an autonomous flight system of a small quadcopter that enabled to track a moving object, where the dynamics of the underactuated robot, the actuator limitations, and the field of view constraints were considered. [28], [29], [30] achieved object tracking for quadcopters by combining an object detection algorithm with an improved PID algorithm. A novel technique based on artificial neural networks was proposed in [31] for the identification and tracking of targets.…”

“…Second, the application scenarios of these tracking systems are limited. For example, some tracking systems are only able to track the ground target [28], [29], [33] or the target with other prior information [26], [27]. This occurs because of the lack of accurate state estimation on the target.…”