A survey of FPGA-based accelerators for convolutional neural networks

Mittal, Sparsh

doi:10.1007/s00521-018-3761-1

Cited by 257 publications

(90 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…With a FPGA it is possible to optimize the datapath, the pipeline structure, the arithmetic units with specific quantizations, the memory hierarchy, structures to support data reduction techniques, etc. Therefore, FPGAs are a good alternative for deep learning inference with good performance and energy [58][59][60][61][62].…”

Section: Fine-grain Reconfigurable Architectures For Cnn On Edgementioning

confidence: 99%

A Survey of Convolutional Neural Networks on Edge with Reconfigurable Computing

Véstias

2019

Algorithms

View full text Add to dashboard Cite

The convolutional neural network (CNN) is one of the most used deep learning models for image detection and classification, due to its high accuracy when compared to other machine learning algorithms. CNNs achieve better results at the cost of higher computing and memory requirements. Inference of convolutional neural networks is therefore usually done in centralized high-performance platforms. However, many applications based on CNNs are migrating to edge devices near the source of data due to the unreliability of a transmission channel in exchanging data with a central server, the uncertainty about channel latency not tolerated by many applications, security and data privacy, etc. While advantageous, deep learning on edge is quite challenging because edge devices are usually limited in terms of performance, cost, and energy. Reconfigurable computing is being considered for inference on edge due to its high performance and energy efficiency while keeping a high hardware flexibility that allows for the easy adaption of the target computing platform to the CNN model. In this paper, we described the features of the most common CNNs, the capabilities of reconfigurable computing for running CNNs, the state-of-the-art of reconfigurable computing implementations proposed to run CNN models, as well as the trends and challenges for future edge reconfigurable platforms.

show abstract

Section: Fine-grain Reconfigurable Architectures For Cnn On Edgementioning

confidence: 99%

A Survey of Convolutional Neural Networks on Edge with Reconfigurable Computing

Véstias

2019

Algorithms

View full text Add to dashboard Cite

show abstract

“…The optical output of the 3DTM is imaged on the charge-coupled device (CCD) camera (output h ð3Þ ), which performs a nonlinear function by measuring the laser intensity. The digital image form the CCD is the input to an electronic pooling convolutional network 35 , which produces the signals denoted as g 1 , g 2 , …, g n . Each of these signals is an average over the pixels on a camera segment.…”

Section: Resultsmentioning

confidence: 99%

Living optical random neural network with three dimensional tumor spheroids for cancer morphodynamics

et al. 2020

View full text Add to dashboard Cite

Optical neural networks process information at the speed of light and are energetically efficient. Photonic artificial intelligence allows speech recognition, image classification, and Ising machines. Modern machine learning paradigms, as extreme learning machines, reveal that disordered and biological materials may realize optical neural networks with thousands of nodes trained only at the input and at the readout. May we use living matter for machine learning? Here, we employ living three-dimensional tumor brain models to demonstrate a random optical learning machine (ROM) for the investigation of glioblastoma. The tumor spheroid act as a computational reservoir. The ROM detects cancer morphodynamics by laser-induced hyperthermia, quantifies chemotherapy, and cell metabolism. The ROM is a sensitive noninvasive smart probe for cytotoxicity assay and enables real-time investigation of tumor dynamics. We hence design and demonstrate a novel bio-hardware for optical computing and the study of light/complex matter interaction.

show abstract

“…An extensive exploration of the merits of leveraging FPGA technology for accelerating CNNs has been conducted and an overview of the most outstanding works has been reported on extended surveys [20]. Works included in these surveys span from optimized HDL CNN implementations to HLS-based designs in the most recent years and even mature CNN-to-FPGA toolflows.…”

Section: Related Workmentioning

confidence: 99%

A TensorFlow Extension Framework for Optimized Generation of Hardware CNN Inference Engines

et al. 2020

View full text Add to dashboard Cite

The workloads of Convolutional Neural Networks (CNNs) exhibit a streaming nature that makes them attractive for reconfigurable architectures such as the Field-Programmable Gate Arrays (FPGAs), while their increased need for low-power and speed has established Application-Specific Integrated Circuit (ASIC)-based accelerators as alternative efficient solutions. During the last five years, the development of Hardware Description Language (HDL)-based CNN accelerators, either for FPGA or ASIC, has seen huge academic interest due to their high-performance and room for optimizations. Towards this direction, we propose a library-based framework, which extends TensorFlow, the well-established machine learning framework, and automatically generates high-throughput CNN inference engines for FPGAs and ASICs. The framework allows software developers to exploit the benefits of FPGA/ASIC acceleration without requiring any expertise on HDL development and low-level design. Moreover, it provides a set of optimization knobs concerning the model architecture and the inference engine generation, allowing the developer to tune the accelerator according to the requirements of the respective use case. Our framework is evaluated by optimizing the LeNet CNN model on the MNIST dataset, and implementing FPGA- and ASIC-based accelerators using the generated inference engine. The optimal FPGA-based accelerator on Zynq-7000 delivers 93% less memory footprint and 54% less Look-Up Table (LUT) utilization, and up to 10× speedup on the inference execution vs. different Graphics Processing Unit (GPU) and Central Processing Unit (CPU) implementations of the same model, in exchange for a negligible accuracy loss, i.e., 0.89%. For the same accuracy drop, the 45 nm standard-cell-based ASIC accelerator provides an implementation which operates at 520 MHz and occupies an area of 0.059 mm 2 , while the power consumption is ∼7.5 mW.

show abstract

A survey of FPGA-based accelerators for convolutional neural networks

Cited by 257 publications

References 61 publications

A Survey of Convolutional Neural Networks on Edge with Reconfigurable Computing

A Survey of Convolutional Neural Networks on Edge with Reconfigurable Computing

Living optical random neural network with three dimensional tumor spheroids for cancer morphodynamics

A TensorFlow Extension Framework for Optimized Generation of Hardware CNN Inference Engines

Contact Info

Product

Resources

About