An FPGA-Based Hardware Accelerator for CNNs Inference on Board Satellites: Benchmarking with Myriad 2-Based Solution for the CloudScout Case Study

Rapuano, Emilio; Meoni, Gabriele; Pacini, Tommaso; Dinelli, Gianmarco; Furano, Gianluca; Giuffrida, Gianluca; Fanucci, Luca

doi:10.3390/rs13081518

Cited by 45 publications

(41 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Here, we briefly introduce some of the most widely used embedded devices and Commercial Off-The-Shelf (COTS) hardware accelerators specifically meant to run NN inferences. [39,40] especially when compared to COTS devices like the ones mentioned above. It is worth noting that recently some FPGA manufacturers have released tools to help developers deploy NNs on their FPGA boards.…”

Section: Boards and Hardware Accelerators For Neural Networkmentioning

confidence: 99%

Oil Spill Identification from SAR Images for Low Power Embedded Systems Using CNN

Diana

Fanucci

2021

Remote Sensing

Self Cite

View full text Add to dashboard Cite

Oil spills represent one of the major threats to marine ecosystems. Satellite synthetic-aperture radar (SAR) sensors have been widely used to identify oil spills due to their ability to provide high resolution images during day and night under all weather conditions. In recent years, the use of artificial intelligence (AI) systems, especially convolutional neural networks (CNNs), have led to many important improvements in performing this task. However, most of the previous solutions to this problem have focused on obtaining the best performance under the assumption that there are no constraints on the amount of hardware resources being used. For this reason, the amounts of hardware resources such as memory and power consumption required by previous solutions make them unsuitable for remote embedded systems such as nano and micro-satellites, which usually have very limited hardware capability and very strict limits on power consumption. In this paper, we present a CNN architecture for semantically segmenting SAR images into multiple classes. The proposed CNN is specifically designed to run on remote embedded systems, which have very limited hardware capability and strict limits on power consumption. Even if the performance in terms of results accuracy does not represent a step forward compared with previous solutions, the presented CNN has the important advantage of being able to run on remote embedded systems with limited hardware resources while achieving good performance. The presented CNN is compatible with dedicated hardware accelerators available on the market due to its low memory footprint and small size. It also provides many additional very significant advantages, such as having shorter inference times, requiring shorter training times, and avoiding transmission of irrelevant data. Our goal is to allow embedded low power remote devices such as satellite systems for remote sensing to be able to directly run CNNs on board, so that the amount of data that needs to be transmitted to ground and processed on ground can be substantially reduced, which will be greatly beneficial in significantly reducing the amount of time needed for identification of oil spills from SAR images.

show abstract

Section: Boards and Hardware Accelerators For Neural Networkmentioning

confidence: 99%

Oil Spill Identification from SAR Images for Low Power Embedded Systems Using CNN

Diana

Fanucci

2021

Remote Sensing

Self Cite

View full text Add to dashboard Cite

show abstract

“…This expression indicates that computations are performed directly where the information source is, without involving data transfers towards external hardware (e.g., cloud server) for the computations [12]. In recent years, the on-the-edge paradigm has acquired primary importance in many fields such as remote sensing [13], autonomous driving [14], or healthcare [15]. Indeed, it ensures advantages in terms of latency, security, and system-required bandwidth (reducing the amount of raw data to be transmitted) [16].…”

Section: Introductionmentioning

confidence: 99%

A Multi-Cache System for On-Chip Memory Optimization in FPGA-Based CNN Accelerators

Pacini

Rapuano

Dinelli³

et al. 2021

Electronics

Self Cite

View full text Add to dashboard Cite

In recent years, FPGAs have demonstrated remarkable performance and contained power consumption for the on-the-edge inference of Convolutional Neural Networks. One of the main challenges in implementing this class of algorithms on board an FPGA is resource management, especially with regard to memory. This work presents a multi-cache system that allows for noticeably shrinking the required on-chip memory with a negligible variation of timing performance and power consumption. The presented methods have been applied to the CloudScout CNN, which was developed to perform cloud detection directly on board the satellite, thus representing a relevant case study for on the edge applications. The system was validated and characterized on a Xilinx ZCU106 Evaluation Board. The result is a 64.48% memory saving if compared to an alternative hardware accelerator developed for the same algorithm, with comparable performance in terms of inference time and power consumption. The paper also presents a detailed analysis of the hardware accelerator power consumption, focusing on the impact of data transfer between the accelerator and the external memory. Further investigation shows that the proposed strategies allow the implementation of the accelerator on FPGAs with a smaller size, guaranteeing benefits in terms of power consumption and hardware costs. A broader evaluation about the applicability of the presented methods to other models demonstrates valuable results in terms of memory saving with respect to other works reported in the literature.

show abstract

“…Space missions for deep space exploration or remote sensing using small satellites faces major difficulties regards energy consumption and communication downlinks for big volumes of data. These difficulties imposes severe restrictions regards automatic decisions on board satellites (Silva & Lucena, 2005a;Silva & Lucena, 2005b;Giuffrida et al, 2020;Rapuano et al, 2021;Furano et al, 2020;Nagel et al, 2020), mainly if they demand digital signal processing from high spatial and spectral resolution sensors (Silva & Lucena, 2005a;Silva & Lucena, 2005b;Giuffrida et al, 2020;Zhou & Kafatos, 2002;Felipe et al, 2006;Thompson et al, 2012;Fuchs et al, 2014;Chien et al, 2014).…”

Section: Introductionmentioning

confidence: 99%

“…One of the most promising proposals to overcome these difficulties is the development of new low power embedded intelligent devices that can provide smart compression method for images and/or that can recognize patterns near real-time using low power consumption on the spacecraft. These intelligent devices also are great promise for more efficiently use regards high-resolution sensors with the restrictions of size, energy consumption and communication bandwidth in space links (Rapuano et al, 2021;Furano et al, 2020;Giuffrida et al, 2020;Felipe et al, 2006;Dawood et al, 2002;Chien et al, 2004;Silva & Lucena, 2005a;Silva & Lucena, 2005b).…”

Section: Introductionmentioning

confidence: 99%

“…However embedded or standalone real-time image processing systems that require low power are not easily implemented using classical processors and sequential processing paradigm (Rapuano et al, 2021;Furano et al, 2020;Giuffrida et al, 2020;Arechiga et al, 2018;Brugger et al 2015;Astua et al, 2014;Blake et al, 2009;Kalomiros & Lygouras, 2008;Akil & Zahirazami, 1998). Related works from specialized literature (Rapuano et al, 2021;Dinelli et al, 2020;Guo et al, 2018;Arechiga et al, 2018;Hentati et al, 2014;Hagiwara et al, 2011;Pell et al, 2013;Zakerhaghighi & Naji, 2013;Bekker et al, 2010;Kalomiros & Lygouras, 2008;Johnston et al, 2004) tried several image processing implementations using FPGA (Field Programmable Gate Array) to build low-power systems for real-time applications.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Pattern recognition on FPGA for aerospace applications

Silva

Filho

Lucena

et al. 2021

RSD

View full text Add to dashboard Cite

This paper presents a low power near real-time pattern recognition technique based on Mathematical Morphology-MM implemented on FPGA (Field Programmable Gate Array). The key to the success of this approach concerns the advantages of machine learning paradigm applied to the translation invariant template-matching operators from MM. The paper shows that compositions of simple elementary operators from Mathematical Morphology based on ELUTs (Elementary Look-Up Tables) are very suitable to embed in FPGA hardware. The paper also shows the development techniques regarding all mathematical modeling for computer simulation and system generating models applied for hardware implementation using FPGA chip. In general, image processing on FPGAs requires low-level description of desired operations through Hardware Description Language-HDL, which uses high complexity to describe image operations at pixel level. However, this work presents a reconfiguring pattern recognition device implemented directly in FPGA from mathematical modeling simulation under Matlab/Simulink/System Generator environment. This strategy has reduced the hardware development complexity. The device will be useful mainly when applied on remote sensing tasks for aerospace missions using passive or active sensors.

show abstract

An FPGA-Based Hardware Accelerator for CNNs Inference on Board Satellites: Benchmarking with Myriad 2-Based Solution for the CloudScout Case Study

Cited by 45 publications

References 39 publications

Oil Spill Identification from SAR Images for Low Power Embedded Systems Using CNN

Oil Spill Identification from SAR Images for Low Power Embedded Systems Using CNN

A Multi-Cache System for On-Chip Memory Optimization in FPGA-Based CNN Accelerators

Pattern recognition on FPGA for aerospace applications

Contact Info

Product

Resources

About