Energy-reliability limits in nanoscale neural networks

Chatterjee, Avhishek; Varshney, Lav R.

doi:10.1109/ciss.2017.7926139

Cited by 6 publications

(3 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that earlier presentations of this work [1], [2] were focused only on circuits with homogeneous electrical operating points for devices, whereas the new synthesis in the current paper emphasizes the value of heterogeneous operation of gates and neural network layers. Results on energy allocation are therefore novel to this paper.…”

Section: A Contributionsmentioning

confidence: 98%

“…So, a necessary condition for order-optimal (linear) energy scaling with a per-neuron energy consumption e G , i.e., for energy consumption scaling as e G N , a necessary condition is that π G (L) must grow exponentially with L with an exponent d G = 1/(1 − 2χ(e g )) 2 . This poses the following constraint on the structure of the deep neural network.…”

Section: A Homogeneous Neuronsmentioning

confidence: 99%

See 1 more Smart Citation

Energy-Reliability Limits in Nanoscale Feedforward Neural Networks and Formulas

Chatterjee

Varshney

2020

IEEE J. Sel. Areas Inf. Theory

Self Cite

View full text Add to dashboard Cite

Due to energy-efficiency requirements, computational systems are now being implemented using noisy nanoscale semiconductor devices whose reliability depends on energy consumed. We study circuit-level energy-reliability limits for deep feedforward neural networks (multilayer perceptrons) built using such devices, and en route also establish the same limits for formulas (boolean tree-structured circuits). To obtain energy lower bounds, we extend Pippenger's mutual information propagation technique for characterizing the complexity of noisy circuits, since small circuit complexity need not imply low energy. Many device technologies require all gates to have the same electrical operating point; in circuits of such uniform gates, we show that the minimum energy required to achieve any non-trivial reliability scales superlinearly with the number of inputs. Circuits implemented in emerging device technologies like spin electronics can, however, have gates operate at different electrical points; in circuits of such heterogeneous gates, we show energy scaling can be linear in the number of inputs. Building on our extended mutual information propagation technique and using crucial insights from convex optimization theory, we develop an algorithm to compute energy lower bounds for any given boolean tree under heterogeneous gates. This algorithm runs in linear time in number of gates, and is therefore practical for modern circuit design. As part of our development we find a simple procedure for energy allocation across circuit gates with different operating points and neural networks with differently-operating layers.

show abstract

Section: A Contributionsmentioning

confidence: 98%

Section: A Homogeneous Neuronsmentioning

confidence: 99%

Energy-Reliability Limits in Nanoscale Feedforward Neural Networks and Formulas

Chatterjee

Varshney

2020

IEEE J. Sel. Areas Inf. Theory

Self Cite

View full text Add to dashboard Cite

show abstract

“…This process is called fault simulation. Fault simulation of DNNs is a time-consuming process as they involve massive computation of a large number of neurons [2].…”

Section: Introductionmentioning

confidence: 99%

Hierarchical Fault Simulation of Deep Neural Networks on Multi-Core Systems

Karami

Haghbayan

Ebrahimi

et al. 2021

2021 IEEE European Test Symposium (ETS)

View full text Add to dashboard Cite

In this paper, a hierarchical fault simulation technique for neural networks is proposed, supporting both permanent and temporary faults. In the proposed technique, different levels of hierarchy are used, forming a mixed-level simulation environment. In such an environment, the pre-synthesis behavioral specification of the network and the post-synthesis gatelevel model are co-simulated. To accelerate the fault simulation process, faults are injected in the gate-level specification of the selected neurons while the behavioral model in different levels of abstraction is used to simulate the remaining neurons. Further speedup is obtained through event-driven simulation and parallelization. Experimental results confirm the time efficiency of the proposed fault simulation technique.

show abstract

Optimal Energy Allocation in Reliable Neural Sensory Processing

Chatterjee

Varshney

2022

Encyclopedia of Computational Neuroscience

View full text Add to dashboard Cite

Energy-reliability limits in nanoscale neural networks

Cited by 6 publications

References 26 publications

Energy-Reliability Limits in Nanoscale Feedforward Neural Networks and Formulas

Energy-Reliability Limits in Nanoscale Feedforward Neural Networks and Formulas

Hierarchical Fault Simulation of Deep Neural Networks on Multi-Core Systems

Optimal Energy Allocation in Reliable Neural Sensory Processing

Contact Info

Product

Resources

About