Efficient Object Detection Framework and Hardware Architecture for Remote Sensing Images

Abstract

Object detection in remote sensing images on a satellite or aircraft has important economic and military significance and is full of challenges. This task requires not only accurate and efficient algorithms, but also highperformance and low power hardware architecture. However, existing deep learning based object detection algorithms require further optimization in small objects detection, reduced computational complexity and parameter size. Meanwhile, the generalpurpose processor cannot achieve better power efficiency, and the previous design of deep learning processor has still potential for mining parallelism. To address these issues, we propose an efficient contextbased feature fusion single shot multibox detector (CBFFSSD) framework, using lightweight MobileNet as the backbone network to reduce parameters and computational complexity, adding feature fusion units and detecting feature maps to enhance the recognition of small objects and improve detection accuracy. Based on the analysis and optimization of the calculation of each layer in the algorithm, we propose efficient hardware architecture of deep learning processor with multiple neural processing units (NPUs) composed of 2D processing elements (PEs), which can simultaneously calculate multiple output feature maps. The parallel architecture, hierarchical onchip storage organization, and the local register are used to achieve parallel processing, sharing and reuse of data, and make the calculation of processor more efficient. Extensive experiments and comprehensive evaluations on the public NWPU VHR10 dataset and comparisons with some stateoftheart approaches demonstrate the effectiveness and superiority of the proposed framework. Moreover, for evaluating the performance of proposed hardware architecture, we implement it on Xilinx XC7Z100 field programmable gate array (FPGA) and test on the proposed CBFFSSD and VGG16 models. Experimental results show that our processor are more power efficient than general purpose central processing units (CPUs) and graphics processing units (GPUs), and have better performance density than other stateoftheart FPGAbased designs.

Patch-Based Three-Stage Aggregation Network for Object Detection in High Resolution Remote Sensing Images

“…Our PTAN also outperformed other object detectors in five categories including airplane, storage tank, basketball court, bridge and vehicle. We note that CBFF-SSD [28] obtained a comparable mAP of 0.9142. However, it used two feature fusion units and seven feature maps, and analyzed the calculation of each layer in the framework.…”

“…We also compared the proposed PTAN framework with other five object detectors including RICNN [21], R-P-Faster RCNN [15], CBFF-SSD [28], MIF-CNN [29] and DFCCNN-VGG [30]. These methods were proposed for remote sensing image object detection.…”

A Lightweight Object Detection Framework for Remote Sensing Images

“…However, with the continuous development of deep learning technology, its performance grows to be excellent, such as faster regions with CNN features (Faster-RCNN) [7], you only look once (YOLO) [8][9][10], single-shot multi-box detector (SSD) [11], and Thun-derNet [12]. Various methods have been proposed by applying deep learning technology to remote sensing image object detection, such as rotation-invariant convolutional neural networks (RICNN) [13], newly trained CNN [13], and context-based feature fusion single shot multi-box detector (CBFF-SSD) [14], which deploys feature fusion methods to improve detection performance. Although the use of deep learning methods for remote sensing image object detection greatly improves the performance of the detectors, its massive computational complexity and extremely large storage space requirements hinder its deployment on satellites.…”

Editorial for Special Issue “Remote Sensing for Target Object Detection and Identification”

“…Another group of papers [3][4][5][6][7][8][9] proposes object detection and recognition approaches that use images (or videos) acquired in the visible and near-infrared (VNIR) wavelength range, making use of the high (or very high) spatial resolution and high spectral content. Indeed, the latter are key features in order to identify shapes, thus enabling more reliable object detection and recognition.…”

“…Another interesting application of CNNs is described in Zhang et al [5], in which vehicle detection for traffic monitoring systems is performed using satellite video data. In contrast, Li et al [6] focus their work on the design of a parallel hardware architecture, based on multiple neural processing units (NPUs), for performing a power-efficient object detection by using CNNs. Liu et al [7] explore alternative frameworks to CNNs with the aim of avoiding time-consuming training phases.…”