Processing Systems for Deep Learning Inference on Edge Devices

Véstias, Mário P.

doi:10.1007/978-3-030-44907-0_9

Cited by 9 publications

(3 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The performance variation is a strong selection criterion as it depends on the technology, data size and area efficiency of the AI/ML design. These different features justify the differences in peak performance among the architectures [95]. Several vendors involved with the design of AI/ML ASIC-based accelerators have selected their own hardware architecture.…”

Section: Asic-based Acceleratorsmentioning

confidence: 99%

Best Practices for the Deployment of Edge Inference: The Conclusions to Start Designing

2021

View full text Add to dashboard Cite

The number of Artificial Intelligence (AI) and Machine Learning (ML) designs is rapidly increasing and certain concerns are raised on how to start an AI design for edge systems, what are the steps to follow and what are the critical pieces towards the most optimal performance. The complete development flow undergoes two distinct phases; training and inference. During training, all the weights are calculated through optimization and back propagation of the network. The training phase is executed with the use of 32-bit floating point arithmetic as this is the convenient format for GPU platforms. The inference phase on the other hand, uses a trained network with new data. The sensitive optimization and back propagation phases are removed and forward propagation is only used. A much lower bit-width and fixed point arithmetic is used aiming a good result with reduced footprint and power consumption. This study follows the survey based process and it is aimed to provide answers such as to clarify all AI edge hardware design aspects from the concept to the final implementation and evaluation. The technology as frameworks and procedures are presented to the order of execution for a complete design cycle with guaranteed success.

show abstract

Section: Asic-based Acceleratorsmentioning

confidence: 99%

Best Practices for the Deployment of Edge Inference: The Conclusions to Start Designing

2021

View full text Add to dashboard Cite

show abstract

“…CNNs in particular have very deep and computationally intensive architectures, but the operations involved are highly modular and repetitive, making them excellent candidates for acceleration through custom hardware. A survey by Véstias et al [49] focuses specifically on accelerating CNNs using reconfigurable computing hardware, while another from Véstias [50] focuses on hardware acceleration of deep learning in general.…”

Section: B Enabling Ai In the Edgementioning

confidence: 99%

Enabling and Leveraging AI in the Intelligent Edge: A Review of Current Trends and Future Directions

Goethals¹,

Volckaert

Turck

2021

IEEE Open J. Commun. Soc.

View full text Add to dashboard Cite

The use of AI in Smart applications and in the organization of the network edge presents a rapidly advancing research field, with a great variety of challenges and opportunities. This article aims to provide a holistic review of studies from 2019 to 2021 related to the Intelligent Edge, a concept comprising both the use of AI to organize edge networks (Edge Intelligence) and Smart applications in the edge. An introduction is given to the technologies required to understand the state of the art of AI in edge networks, and a taxonomy is provided with "Enabling Technology" for Edge Intelligence, "Organization" of the edge using AI, and AI "Applications" in the edge as its main topics. Research trend data from 2015 to 2020 is presented for various subdivisions of these topics, showing both absolute and relative research interest in each subtopic. The "Organization" aspect, being the main focus of this article, has a more fine-grained subdivision, explaining all contributing factors in detail. The trends indicate an exponential increase in research interest in nearly all subtopics, but significant differences between them. For each subdivision of the taxonomy a number of selected studies from 2019 to 2021 are gathered to form a high-level illustration of the state of the art of Edge Intelligence. From these selected studies and the trend data, a number of short-term challenges and high-level visions for Edge Intelligence are formulated, providing a basis for future work.

show abstract

“…Computing platforms for deep learning on edge cannot rely on high-performance devices since they either have high energy consumption or low power efficiency and are relatively expensive [11]. Embedded processors are used in many low cost devices but achieve only a few dozen GFLOPs (Giga FLoating-point Operations Per second) with low power efficiency, insufficient for real or almost realtime processing of CNNs.…”

Section: Related Workmentioning

confidence: 99%

Efficient Design of Pruned Convolutional Neural Networks on FPGA

Véstias

2020

J Sign Process Syst

Self Cite

View full text Add to dashboard Cite

Convolutional Neural Networks (CNNs) have improved several computer vision applications, like object detection and classification, when compared to other machine learning algorithms. Running these models in edge computing devices close to data sources is attracting the attention of the community since it avoids high-latency data communication of private data for cloud processing and permits real-time decisions turning these systems into smart embedded devices. Running these models is computationally very demanding and requires a large amount of memory, which are scarce in edge devices compared to a cloud center. In this paper, we proposed an architecture for the inference of pruned convolutional neural networks in any density FPGAs. A configurable block pruning method is proposed together with an architecture that supports the efficient execution of pruned networks. Also, pruning and batching are studied together to determine how they influence each other. With the proposed architecture, we run the inference of a CNN with an average performance of 322 GOPs for 8-bit data in a XC7Z020 FPGA. The proposed architecture running AlexNet processes 240 images/s in a ZYNQ7020 and 775 images/s in a ZYNQ7045 with only 1.2% accuracy degradation.

show abstract

Processing Systems for Deep Learning Inference on Edge Devices

Cited by 9 publications

References 26 publications

Best Practices for the Deployment of Edge Inference: The Conclusions to Start Designing

Best Practices for the Deployment of Edge Inference: The Conclusions to Start Designing

Enabling and Leveraging AI in the Intelligent Edge: A Review of Current Trends and Future Directions

Efficient Design of Pruned Convolutional Neural Networks on FPGA

Contact Info

Product

Resources

About