Towards an Efficient Accelerator for DNN-Based Remote Sensing Image Segmentation on FPGAs

Abstract

Among popular techniques in remote sensing image (RSI) segmentation, Deep Neural Networks (DNNs) have gained increasing interest but often require high computation complexity, which largely limit their applicability in on-board space platforms. Therefore, various dedicated hardware designs on FPGAs have been developed to accelerate DNNs. However, it imposes difficulty on the design of efficient accelerator for DNN-based segmentation algorithms, since they need to perform both convolution and deconvolution which are two fundamentally different types of operations. This paper proposes a uniform architecture to efficiently implement both convolution and deconvolution in one vector multiplication module. This architecture is further optimized through exploiting different levels of parallelism and layer fusion to achieve low latency for RSI segmentation tasks. Moreover, an optimized DNN model is developed for real-time RSI segmentation, which shows preferable accuracy compared to other methods. The proposed hardware accelerator efficiently implements the DNN model on Intel's Arria 10 device, demonstrating 1578 GOPS of throughput and 17.4 ms of latency, i.e., 57 images per second.

Efficient Biomedical Image Segmentation on EdgeTPUs at Point of Care

“…Porting a neural network to hardware accelerators has been performed for various architectures [27]. This is especially common for FPGAs [28], [29], and used for example in hand tracking [30] or language processing [31]. Special attention is also on adapting key principles in neural network archi- tectures, such as depth-wise convolutions for FPGAs [32] or quantized-based operations, such as binary neural networks [33].…”

Binarized Encoder-Decoder Network and Binarized Deconvolution Engine for Semantic Segmentation

“…The main difference between the traditional classification networks and the segmentation networks is the deconvolution layer, so these studies focused on the acceleration of deconvolution, presenting hardware architectures for padding free deconvolution or zero padding deconvolution. Liu et al [20] and FCN-engine [23] accelerated the padding free deconvolution of an 8-bit network with FPGA and application specific integrated circuit (ASIC), respectively. Liu et al inferred the 256 × 256 images at 1.79 FPS/W.…”

“…The resulting power consumption and latency are expected to be large, but existing studies have not considered it [21]- [23], [37], [39], [41]. Liu et al [20] and Song et al [38] store all feature maps in on-chip memory. This was possible because the former significantly reduced the image size and network channel size, and the latter was GAN, the input image and network channel size being small.…”

“…It is noteworthy that as the number of SIMD lanes increased, the performance efficiency (GOPS/W) also improved. Since the FPGA employed in the experiment cannot accommodate more SIMD lanes, the experiment requiring more SIMD lanes could not be conducted, but higher performance efficiency is expected We also compared the performance of BiDE with Liu et al [20], an existing 8-bit segmentation network accelerator, and the results are shown in Table. 8. BiDE carried out the inferences of 480 × 360 images of the CamVid11 dataset at 25.89 FPS with 2 watt (W) power consumption, and achieved 1682.85 logical GOPS and 1190.94 physical GOPS.…”

“…Furthermore, in the case of quad configuration, 70% of the LUTs were utilized, but it was the maximum configuration that could be implemented on the FPGA employed in the experiments because more logic could not be used due to routing problems. In Table. 9, the FPGA resource utilization of the quad configuration of BiDE and that of Liu et al [20] are compared. BiDE and Liu et al utilized similar amounts of FFs and BRAMs.…”