Fault resilient physical neural networks on a single chip

Abstract

Device scaling engineering is facing major challenges in producing reliable transistors for future electronic technologies. With shrinking device sizes, the total circuit sensitivity to both permanent and transient faults has increased significantly. Research for fault tolerant processors has primarily focused on the conventional processor architectures. Neural network computing has been employed to solve a wide range of problems. This paper presents a design and implementation of a physical neural network that is resilient to permanent hardware faults. To achieve scalability, it uses tiled neuron clusters where neuron outputs are efficiently forwarded to the target neurons using source based spanning tree routing. To achieve fault resilience in the face of increasing number of permanent hardware failures, the design proactively preserves neural network computing performance by selectively replicating performance critical neurons. Furthermore, the paper presents a spanning tree recovery solution that mitigates disruption to distribution of neuron outputs caused by failed neuron clusters. The proposed neuron cluster design is implemented in Verilog. We studied the fault resilience performance of the described design using a RBM neural network trained for classifying handwritten digit images. Results demonstrate that our approach can achieve improved fault resilience performance by replicating only 5% most important neurons.

Eden

“…Approximate Computing Hardware for DNN Workloads. Many prior works propose to use approximate computing hardware for executing machine learning workloads [30,38,97,110,118,131,132,134,141,143,155,163,166,169,173,174,179,180]. All these works propose techniques for improving DNN tolerance for different types of approximate hardware mechanisms and error injection rates.…”

“…Fifth, works that study the effects of approximate storage devices on DNN workloads [132,155]. Qin et al [132] study the error tolerance of neural networks that are stored in approximate non-volatile memory (NVM) media.…”

“…The authors study the effects of turning the ECC off in parts of the NVM media that store the neural network data. Wen et al [155] propose to mitigate the effects of unreliable disk reads with a specialized ECC variant that aims to mitigate error patterns present in weights of shallow neural networks.…”