Sign-Magnitude SC: Getting 10X Accuracy for Free in Stochastic Computing for Deep Neural Networks

Zhakatayev, Aidyn; Lee, Sugil; Sim, Hyeonuk; Lee, Jong-Eun

doi:10.1109/dac.2018.8465807

Cited by 27 publications

(15 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For an inference input, we instantiate k such operations, where k is the number of classes. Table 2 compares the baseline accuracy (32-bit integer values) and the quality loss of the applications running on COSMO using 32-bit SM-SC [91] encoding. Our evaluation shows that COSMO can result only about 1.5% and 1% quality loss on DNN and HD computing.…”

Section: Methodsmentioning

confidence: 99%

“…There has been some interest in implementing DNNs using SC [10,43,56,59,75,91]. They provide high performance but that performance comes at the cost of huge silicon area.…”

Section: Deep Neural Network Inferencementioning

confidence: 99%

“…We compare COSMO with GPU for the DNNs and HD computing workloads detailed in Table 2. We use SM-SC encoding with a bit-stream length of 32 [91] to represent the inputs and weights in DNNs and value of each dimension in HD computing on COSMO. Also, evaluation is performed while keeping the COSMO area and technology node the same as GPU.…”

Section: Learning On Cosmo and Gpu Cosmo Configurationsmentioning

confidence: 99%

“…Some try to improve the efficiency of SC operations by proposing new approximate implementations [44,67,87]. The work in [15,59,91] propose new encoding schemes for SC which are more accurate than traditional encoding. Some work also optimize SC for different applications [43,54,59,75,81,91].…”

mentioning

confidence: 99%

“…The work in [15,59,91] propose new encoding schemes for SC which are more accurate than traditional encoding. Some work also optimize SC for different applications [43,54,59,75,81,91].…”

mentioning

confidence: 99%

See 4 more Smart Citations

COSMO: Computing with Stochastic Numbers in Memory

Gupta

Imani

Sim

et al. 2022

J. Emerg. Technol. Comput. Syst.

View full text Add to dashboard Cite

Stochastic computing (SC) reduces the complexity of computation by representing numbers with long streams of independent bits. However, increasing performance in SC comes with either an increase in area or a loss in accuracy. Processing in memory (PIM) computes data in-place while having high memory density and supporting bit-parallel operations with low energy consumption. In this article, we propose COSMO, an architecture for co mputing with s tochastic numbers in me mo ry, which enables SC in memory. The proposed architecture is general and can be used for a wide range of applications. It is a highly dense and parallel architecture that supports most SC encodings and operations in memory. It maximizes the performance and energy efficiency of SC by introducing several innovations: (i) in-memory parallel stochastic number generation, (ii) efficient implication-based logic in memory, (iii) novel memory bit line segmenting, (iv) a new memory-compatible SC addition operation, and (v) enabling flexible block allocation. To show the generality and efficiency of our stochastic architecture, we implement image processing, deep neural networks (DNNs), and hyperdimensional (HD) computing on the proposed hardware. Our evaluations show that running DNN inference on COSMO is 141× faster and 80× more energy efficient as compared to GPU.

show abstract

Section: Methodsmentioning

confidence: 99%

“…There has been some interest in implementing DNNs using SC [10,43,56,59,75,91]. They provide high performance but that performance comes at the cost of huge silicon area.…”

Section: Deep Neural Network Inferencementioning

confidence: 99%

Section: Learning On Cosmo and Gpu Cosmo Configurationsmentioning

confidence: 99%

mentioning

confidence: 99%

“…The work in [15,59,91] propose new encoding schemes for SC which are more accurate than traditional encoding. Some work also optimize SC for different applications [43,54,59,75,81,91].…”

mentioning

confidence: 99%

See 3 more Smart Citations