Dynamic energy-accuracy trade-off using stochastic computing in deep neural networks

Kim, Kyounghoon; Kim, Jungki; Yu, Jeesuk; Seo, Ji Hoon; Lee, Jong-Eun; Choi, Ki‐Young

doi:10.1145/2897937.2898011

Cited by 166 publications

(101 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It provides many benefits for neural networks such as low computation footprint, error tolerance, simple implementation in circuits and better trade-off be-tween time and accuracy [89]. Many works contribute to exploring potential space in optimization and in deep belief networks [90,91,92]. Recently it starts to gain attentions in CNN field and regarded as a promising technique for deep CNN implementation on ASIC (Section 6.3) and on embedded portable devices as it can significantly reduce resource consumption with high accuracy.…”

Section: Redundancy In Representationsmentioning

confidence: 99%

Recent advances in convolutional neural network acceleration

et al. 2019

View full text Add to dashboard Cite

In recent years, convolutional neural networks (CNNs) have shown great performance in various fields such as image classification, pattern recognition, and multi-media compression. Two of the feature properties, local connectivity and weight sharing, can reduce the number of parameters and increase processing speed during training and inference. However, as the dimension of data becomes higher and the CNN architecture becomes more complicated, the endto-end approach or the combined manner of CNN is computationally intensive, which becomes limitation to CNN's further implementation. Therefore, it is necessary and urgent to implement CNN in a faster way. In this paper, we first summarize the acceleration methods that contribute to but not limited to CNN by reviewing a broad variety of research papers. We propose a taxonomy in terms of three levels, i.e. structure level, algorithm level, and implementation level, for acceleration methods. We also analyze the acceleration methods in terms of CNN architecture compression, algorithm optimization, and hardware-based improvement. At last, we give a discussion on different perspectives of these acceleration and optimization methods within each level. The discussion shows that the methods in each level still have large exploration space. By incorporating such a wide range of disciplines, we expect to provide a comprehensive reference for researchers who are interested in CNN acceleration.

show abstract

Section: Redundancy In Representationsmentioning

confidence: 99%

Recent advances in convolutional neural network acceleration

et al. 2019

View full text Add to dashboard Cite

show abstract

“…where (·) denotes the negation operator. Please note that (8) corresponds to the Boolean function of the CC. In particular, the canceling procedure is as follows: Upon a rising edge of the higher clock, the CC output is written into the next register element.…”

Section: Stochastic Inner Product Desginmentioning

confidence: 99%

High-Accuracy and Fault Tolerant Stochastic Inner Product Design

Haselmayr

Wiesinger

Lunglmayr

2020

IEEE Trans. Circuits Syst. II

View full text Add to dashboard Cite

In this work, we present a novel inner product design for stochastic computing. Stochastic computing is an emerging computing technique, that encodes a number in the probability of observing a one in a random bit stream. This leads to reduced hardware costs and high error tolerance. The proposed inner product design is based on a two-line bipolar encoding format and applies sequential processing of the input in a central accumulation unit. Sequential processing significantly increases the computation accuracy, since it allows for preliminary cancelation of carry bits. Moreover, the central accumulation unit gives a much better scalability compared to conventional adder tree approaches. We show that the proposed inner product design outperforms a state-of-the-art design in terms of hardware costs for high accuracy requirements and fault tolerance.

show abstract

“…Similarly, addition can be implemented with a MUX gate. Therefore, SC is successfully applied to computation-intensive applications such as digital signal processing [8], artificial neural networks (ANN) [9] and decoding of modern error-correcting codes [10]. Another inherent feature of SC is high error tolerance [4], a single bit-flip in a long bitstream only causes a small change of value and different flips tend to cancel each other out [11].…”

Section: A Stochastic Computing (Sc)mentioning

confidence: 99%

Memory System Designed for Multiply-Accumulate (MAC) Engine Based on Stochastic Computing

Zhang

Wang

Zhang

et al. 2019

2019 International Conference on IC Design and Technology (ICICDT)

View full text Add to dashboard Cite

Convolutional neural network (CNN) achieves excellent performance on fascinating tasks such as image recognition and natural language processing at the cost of high power consumption. Stochastic computing (SC) is an attractive paradigm implemented in low power applications which performs arithmetic operations with simple logic and low hardware cost. However, conventional memory structure designed and optimized for binary computing leads to extra data conversion costs, which significantly decreases the energy efficiency. Therefore, a new memory system designed for SC-based multiply-accumulate (MAC) engine applied in CNN which is compatible with conventional memory system is proposed in this paper. As a result, the overall energy consumption of our new computing structure is 0.91pJ, which is reduced by 82.1% compared with the conventional structure, and the energy efficiency achieves 164.8 TOPS/W. I.

show abstract

Dynamic energy-accuracy trade-off using stochastic computing in deep neural networks

Cited by 166 publications

References 11 publications

Recent advances in convolutional neural network acceleration

Recent advances in convolutional neural network acceleration

High-Accuracy and Fault Tolerant Stochastic Inner Product Design

Memory System Designed for Multiply-Accumulate (MAC) Engine Based on Stochastic Computing

Contact Info

Product

Resources

About