Efficient ConvNet-based Object Detection for Unmanned Aerial Vehicles by Selective Tile Processing

Abstract

Many applications utilizing Unmanned Aerial Vehicles (UAVs) require the use of computer vision algorithms to analyze the information captured from their on-board camera. Recent advances in deep learning have made it possible to use single-shot Convolutional Neural Network (CNN) detection algorithms that process the input image to detect various objects of interest. To keep the computational demands low these neural networks typically operate on small image sizes which, however, makes it di cult to detect small objects. is is further emphasized when considering UAVs equipped with cameras where due to the viewing range, objects tend to appear relatively small. is paper therefore, explores the trade-o s involved when maintaining the resolution of the objects of interest by extracting smaller patches (tiles) from the larger input image and processing them using a neural network. Speci cally, we introduce an a ention mechanism to focus on detecting objects only in some of the tiles and a memory mechanism to keep track of information for tiles that are not processed. rough the analysis of di erent methods and experiments we show that by carefully selecting which tiles to process we can considerably improve the detection accuracy while maintaining comparable performance to CNNs that resize and process a single image which makes the proposed approach suitable for UAV applications.

Deep Learning for Edge Computing: Current Trends, Cross-Layer Optimizations, and Open Research Challenges

“…In particular, when a high resolution image is resized there is a significant loss of information, where the pixels representing Fig. 5: Comparison between predictions using DroNet [36] with STP [53] (Left) and Resizing (Right).…”

“…To this end, we proposed a Selective Tile Processing (STP) [53] approach where the input image is separated into smaller regions, called tiles, in order to avoid resizing the input image and to maintain the object resolution. Given an input image and the DNN input, the number of tiles are extracted and are uniformly distributed across the input image while maintaining a constant overlap between the tiles.…”

“…The second mechanism is an Attention Mechanism, which selects the tiles that need to be processed by the DNN on the next frame. Using both memory and attention mechanism, we combine gathered information from previous frames, such as the number of objects detected in each tile, to select one or more tiles for processing by selecting the tiles with the highest weighted score presented in [53]. Fig.…”

Informed Region Selection for Efficient UAV-based Object Detectors: Altitude-aware Vehicle Detection with CyCAR Dataset

“…Recent research has proposed pushing cropped overlapping tiles of the input image through the CNN independently, without degrading the initial resolution [20] [21] [22]. However, this approach introduces notable computational overhead that can only be partly alleviated by the use of attention and memory mechanisms for selective tile processing [23].…”

Reliability-Aware Task Allocation Latency Optimization in Edge Computing

“…This task can only be performed by the UAV's camera. Next, the two central parts of this application follow; the power transmission lines detection process, and a convolutional neural network (CNN) based approach [8] for the power tower detection process. The final task is responsible to display the results to the ground operator, therefore, it can only be executed on the ground station.…”