A Multifault‐Tolerant Training Scheme for Nonideal Memristive Neural Networks

Abstract

Memristor crossbar is extensively investigated as an energy‐efficient accelerator for neural network (NN) computations. However, hardware implementation of NNs using realistic memristors is challenging due to the ubiquity of faults (mainly classified into hard and soft faults) in memristors. Herein, a hardware‐friendly, low‐power multifault‐tolerant training (MFTT) scheme capable of addressing both hard and soft faults simultaneously for memristive NNs is proposed. The MFTT scheme consists of multifault detection, targeted weight pruning, and in situ training with the Manhattan update rule. Specifically, multifault detection is first conducted to detect both hard and large soft faults. The detected faulty weights are subsequently pruned to prevent them from disturbing the NN. The sparsified NN after pruning is in situ trained so that the hard and large soft faults can be effectively tolerated using the sparsity and self‐adaptivity of NNs. In addition, the remaining small soft faults can be well tolerated by the Manhattan update rule. Experimentally, MFTT demonstrates the lowest accuracy losses among several representative fault‐tolerant schemes not only in the hard fault‐only (when the hard fault ratio exceeds 10%) and soft fault‐only (when the soft faults are large) cases, but also in the case where both types of faults coexist.

Text classification in memristor-based spiking neural networks

“…To explore how memristor non-ideality affects the system performance, we used (1) an empirical memristor model with non-linear switching dynamics to represent the memristor non-linearity, (2) the R tolerance to represent the write errors during the weight updating process, and (3) the read noise to represent the noise disturbance during the read operation. Naturally, there are other non-ideal effects such as stuck-at faults [94] and the line resistance [95] that arise when utilising memristor-based SNNs. We did not study these factors in this work to keep the focus on algorithm-level simulation including 'non-ideal but functioning devices' only.…”