Hardware–Software Approximations for Deep Neural Networks

Hanif, Muhammad Abdullah; Javed, Muhammad Usama; Hafiz, Rehan; Rehman, Semeen; Shafique, Muhammad

doi:10.1007/978-3-319-99322-5_13

Cited by 5 publications

(7 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Extracting meaningful knowledge from a single input sample can require enormous MAC operations. e number of MAC operations can reach the magnitude of billion [42]. Additionally, a single deep learning network can contain over a million parameters [43].…”

Section: Challenges To Be Investigatedmentioning

confidence: 99%

See 1 more Smart Citation

Deep Learning on Computational-Resource-Limited Platforms: A Survey

Chen

Zhang

et al. 2020

Mobile Information Systems

View full text Add to dashboard Cite

Nowadays, Internet of Things (IoT) gives rise to a huge amount of data. IoT nodes equipped with smart sensors can immediately extract meaningful knowledge from the data through machine learning technologies. Deep learning (DL) is constantly contributing significant progress in smart sensing due to its dramatic superiorities over traditional machine learning. The promising prospect of wide-range applications puts forwards demands on the ubiquitous deployment of DL under various contexts. As a result, performing DL on mobile or embedded platforms is becoming a common requirement. Nevertheless, a typical DL application can easily exhaust an embedded or mobile device owing to a large amount of multiply and accumulate (MAC) operations and memory access operations. Consequently, it is a challenging task to bridge the gap between deep learning and resource-limited platforms. We summarize typical applications of resource-limited deep learning and point out that deep learning is an indispensable impetus of pervasive computing. Subsequently, we explore the underlying reasons for the high computational overhead of DL through reviewing the fundamental concepts including capacity, generalization, and backpropagation of a neural network. Guided by these concepts, we investigate on principles of representative research works, as well as three types of solutions: algorithmic design, computational optimization, and hardware revolution. In pursuant to these solutions, we identify challenges to be addressed.

show abstract

Section: Challenges To Be Investigatedmentioning

confidence: 99%

“…As a result, deep learning proposes high demands on processing ability, memory capacity, and energy efficiency. It is a vital issue to optimize deep learning networks by eliminating ineffectual MAC operations and parameters [42]. e second barrier is fitting DNNs into diversified modern hardware platforms.…”

Section: Challenges To Be Investigatedmentioning

confidence: 99%

Deep Learning on Computational-Resource-Limited Platforms: A Survey

Chen

Zhang

et al. 2020

Mobile Information Systems

View full text Add to dashboard Cite

show abstract

“…For instance, the pioneering DNN in the ImageNet challenge, which surpassed human-level accuracy and is known as the ResNet model, with 152 layers, necessitates 11.3 GMAC operations and 60 million weights [9,10]. Typically, processing a single input sample in a DNN demands approximately one billion MAC operations [11]. This highlights the potential for substantial reductions in computational demands by enhancing the efficiency of MAC operations.…”

Section: Introductionmentioning

confidence: 99%

“…This implies that precision in output is crucial only when the input is a positive value. The input to a ReLU function typically originates from the output of a fully connected or convolution layer in the DNN, involving a substantial number of MAC operations [11]. It is indicated by various researchers that a significant proportion, ranging from 50% to 95%, of ReLU inputs in DNNs are negative [20][21][22][23].…”

Section: Introductionmentioning

confidence: 99%

ECHO: Energy-Efficient Computation Harnessing Online Arithmetic—An MSDF-Based Accelerator for DNN Inference

Ibrahim,

Usman,

Lee

2024

Electronics

View full text Add to dashboard Cite

Deep neural network (DNN) inference demands substantial computing power, resulting in significant energy consumption. A large number of negative output activations in convolution layers are rendered zero due to the invocation of the ReLU activation function. This results in a substantial number of unnecessary computations that consume significant amounts of energy. This paper presents ECHO, an accelerator for DNN inference designed for computation pruning, utilizing an unconventional arithmetic paradigm known as online/most significant digit first (MSDF) arithmetic, which performs computations in a digit-serial manner. The MSDF digit-serial computation of online arithmetic enables overlapped computation of successive operations, leading to substantial performance improvements. The online arithmetic, coupled with a negative output detection scheme, facilitates early and precise recognition of negative outputs. This, in turn, allows for the timely termination of unnecessary computations, resulting in a reduction in energy consumption. The implemented design has been realized on the Xilinx Virtex-7 VU3P FPGA and subjected to a comprehensive evaluation through a rigorous comparative analysis involving widely used performance metrics. The experimental results demonstrate promising power and performance improvements compared to contemporary methods. In particular, the proposed design achieved average improvements in power consumption of up to 81%, 82.9%, and 40.6% for VGG-16, ResNet-18, and ResNet-50 workloads compared to the conventional bit-serial design, respectively. Furthermore, significant average speedups of 2.39×, 2.6×, and 2.42× were observed when comparing the proposed design to conventional bit-serial designs for the VGG-16, ResNet-18, and ResNet-50 models, respectively.

show abstract

“…For image classification, deep convolutional neural networks (CNNs), trained on large datasets such as Ima-geNet2012 [1] are the gold standard. However, even after a CNN has been trained, using it for inference requires on the order of millions (if not billions) of multiply-accumulate operations (see Hanif et al [2], figure 13.5) to propagate real-valued unit activations through many layers of trained weights. This results in computation and energy [3] requirements that make them unsuitable for many mobile and edge-computing applications and raises concerns about the carbon footprint of neural network algorithms.…”

Section: Introductionmentioning

confidence: 99%

mlGeNN: accelerating SNN inference using GPU-enabled neural networks

Turner¹,

Knight²,

Subramanian³

et al. 2022

Neuromorph. Comput. Eng.

View full text Add to dashboard Cite

In this paper we present mlGeNN – a Python library for the conversion of artiﬁcial neural networks (ANNs) speciﬁed in Keras to spiking neural networks (SNNs). SNNs are simulated using GeNN with extensions to eﬃciently support convolutional connectivity and batching. We evaluate converted SNNs on CIFAR-10 and ImageNet classiﬁcation tasks and compare the performance to both the original ANNs and other SNN simulators. We ﬁnd that performing inference using a VGG-16 model, trained on the CIFAR-10 dataset, is 2.5× faster than BindsNet and, when using a ResNet-20 model trained on CIFAR-10 with FewSpike ANN to SNN conversion, mlGeNN is only a little over 2× slower than TensorFlow.

show abstract

Hardware–Software Approximations for Deep Neural Networks

Cited by 5 publications

References 29 publications

Deep Learning on Computational-Resource-Limited Platforms: A Survey

Deep Learning on Computational-Resource-Limited Platforms: A Survey

ECHO: Energy-Efficient Computation Harnessing Online Arithmetic—An MSDF-Based Accelerator for DNN Inference

mlGeNN: accelerating SNN inference using GPU-enabled neural networks

Contact Info

Product

Resources

About