FPGA Implementation for CNN-Based Optical Remote Sensing Object Detection

Abstract

In recent years, convolutional neural network (CNN)-based methods have been widely used for optical remote sensing object detection and have shown excellent performance. Some aerospace systems, such as satellites or aircrafts, need to adopt these methods to observe objects on the ground. Due to the limited budget of the logical resources and power consumption in these systems, an embedded device is a good choice to implement the CNN-based methods. However, it is still a challenge to strike a balance between performance and power consumption. In this paper, we propose an efficient hardware-implementation method for optical remote sensing object detection. Firstly, we optimize the CNN-based model for hardware implementation, which establishes a foundation for efficiently mapping the network on a field-programmable gate array (FPGA). In addition, we propose a hardware architecture for the CNN-based remote sensing object detection model. In this architecture, a general processing engine (PE) is proposed to implement multiple types of convolutions in the network using the uniform module. An efficient data storage and access scheme is also proposed, and it achieves low-latency calculations and a high memory bandwidth utilization rate. Finally, we deployed the improved YOLOv2 network on a Xilinx ZYNQ xc7z035 FPGA to evaluate the performance of our design. The experimental results show that the performance of our implementation on an FPGA is only 0.18% lower than that on a graphics processing unit (GPU) in mean average precision (mAP). Under a 200 MHz working frequency, our design achieves a throughput of 111.5 giga-operations per second (GOP/s) with a 5.96 W on-chip power consumption. Comparison with the related works demonstrates that the proposed design has obvious advantages in terms of energy efficiency and that it is suitable for deployment on embedded devices.

An Efficient Dual-Channel Data Storage and Access Method for Spaceborne Synthetic Aperture Radar Real-Time Processing

“…This paper also compares several existing FPGA-based implementations to verify the superiority of the data controller proposed. The method proposed can also be extended to other two-dimensional image real-time processing scenarios, such as FPGA-based Uninhabited Aerial Vehicle Synthetic Aperture Radar (UAVSAR) [18,39] and Convolutional Neural Networks (CNNs) [40][41][42], etc. In the future, we will conduct research on multichannel SAR data processing, multi-mode integrated SAR processing, and new efficient algorithms that are suitable for real-time processing in order to meet new remote sensing application requirements.…”

Automatic Deployment of Convolutional Neural Networks on FPGA for Spaceborne Remote Sensing Application

“…After the operation unification, an ordered structure of convolution-BN-LeakyReLU is used in a CNN. Inspired by our previous work [62], the quantization operation and inverse quantization operation can be integrated into the ordered computational layers. Notably, if CNN does not include BN operation, the BN operation is implemented with γ = 1 and β = 0 in Equation ( 3) to achieve the convolution-BN-LeakyReLU structure.…”

Deep Convolutional Neural Networks for Weeds and Crops Discrimination From UAS Imagery

“…Some studies prove that deep convolutional neural networks (CNNs) are efficient methods to deal with the limitations of handcrafted features on classifying weeds, crops and seeds (Lee et al, 2015;Loddo et al, 2021). In recent times, there has been considerable progress in the classification and segmentation of remote sensing data using deep learning for different applications (Chen et al, 2020 andZhang et al, 2021). Unlike the conventional ML approaches, CNNs have become an increasingly popular approach for remote sensing tasks due to their ability to extracts and learns feature representation directly from big datasets (Ma et al, 2020).…”