Computational memory-based inference and training of deep neural networks

Abstract

In-memory computing is an emerging computing paradigm where certain computational tasks are performed in place in a computational memory unit by exploiting the physical attributes of the memory devices. Here, we present an overview of the application of in-memory computing in deep learning, a branch of machine learning that has significantly contributed to the recent explosive growth in artificial intelligence. The methodology for both inference and training of deep neural networks is presented along with experimental results using phase-change memory (PCM) devices.

Applications of Phase Change Materials in Electrical Regime From Conventional Storage Memory to Novel Neuromorphic Computing

“…Additionally, the conductance changes in 90 nm Lance-type PCRAM cells was investigated in terms of the granularity and stochasticity. The standard deviation of the conductance change was reportedly similar to the mean conductance change, and the classification accuracy was impaired considerably with the increased size of the mean conductance change [168]. These findings make it possible to innovate the device technologies and synaptic architecture that are more robust to these undesirable properties.…”

“…The test accuracy after 20 epochs of the training was approximately 98%. Besides, another method to train the ResNet-type convolutional neural networks, which leads to almost no accuracy loss when transferring weights to analog in-memory computing hardware based on the phase-change memory, was proposed [167], [168], as schematically shown in Figures 24(f)-(h). The as-programmed classification accuracy of 93.69% on the CIFAR-10 dataset with ResNet-32 was demonstrated based on a network of 361,722 synaptic weights programmed on two phase-change devices deployed in a differential configuration, which stays above 92.6% over a one day period.…”

“…A hardware model incorporating PCM conductance drift (ν = 0.06), device-to-device drift exponent variability with 0.1ν standard deviation, and instantaneous Gaussian read noise with 0.6 µS standard deviation is able to capture the experimental results fairly well. (a)-(e) are reprinted with permission from [166]; (f) is reprinted with permission from [168]; (g) and (h) are reprinted with permission from [167].…”

In-Memory Logic Operations and Neuromorphic Computing in Non-Volatile Random Access Memory

“…Further recent developments in PCM-based computational RAM at IBM have demonstrated the potential of PCRAM to store synaptic weights, taking things a step closer to brain-like memory and processing [132,138]. This technology has been shown to be able to handle cloud-based two-layer neural network processing of relatively large bodies of data [132] and seems to also have the potential to handle more complex, convolutional neural networks [139].…”

“…Several hypotheses have been put forward regarding the RS mechanism of these materials, including valence change memory (VCM) [140][141][142], electrochemical metallization memory (ECMM) [143][144][145], and thermochemical memory [146][147][148] (see Figure 16). two-layer neural network processing of relatively large bodies of data [132] and seems to also have the potential to handle more complex, convolutional neural networks [139]. The challenges confronting the ongoing development of PCRAM and its uptake are, as with MRAM, the relative immaturity of many of the most promising technological approaches and, closely related to this, the currently high cost of the materials required for its implementation.…”