A New Stochastic Computing Methodology for Efficient Neural Network Implementation

Abstract

This paper presents a new methodology for the hardware implementation of neural networks (NNs) based on probabilistic laws. The proposed encoding scheme circumvents the limitations of classical stochastic computing (based on unipolar or bipolar encoding) extending the representation range to any real number using the ratio of two bipolar-encoded pulsed signals. Furthermore, the novel approach presents practically a total noise-immunity capability due to its specific codification. We introduce different designs for building the fundamental blocks needed to implement NNs. The validity of the present approach is demonstrated through a regression and a pattern recognition task. The low cost of the methodology in terms of hardware, along with its capacity to implement complex mathematical functions (such as the hyperbolic tangent), allows its use for building highly reliable systems and parallel computing.

Scalable Stochastic Number Duplicators for Accuracy-flexible Arithmetic Circuit Design

“…For example, an AND gate is used for multiplication, a MUX circuit for weighted addition and a NOT gate for inversion [4]. Hence, SC can be implemented with smaller sized circuits compared with binary computing and has been applied to many hardware applications such as neural networks [2], [3] and image processing [1], [13]. In addition, SC has noise resiliency and have been applied to reliable systems [5], [8], [12], [14], [15], [17], [19], [20].…”

Estimation of Mathematical Function by Stochastic Logic

“…Their future work is completely focused on stochastic design of a microprocessor. Armin Alaghi et al [4] this paper surveys the key concepts of stochastic number representation and circuit structure. It gives examples of the potential applications of SC and showed some practical problems that are yet to be solved.…”

Peer Review #1 of "Stochastic computing in convolutional neural network implementation: a review (v0.1)"

“…Extended stochastic logic (ESL): another radical approach ESL made an extreme modification to the SC methodology if integral SC is not radical enough. Instead of using a single stochastic bitstream for a value, ESL used two stochastic streams such that their ratio of division represents the actual value (Canals et al, 2016). ESL intends to compute the entire range of real numbers in the stochastic domain.…”