Long Live TIME: Improving Lifetime and Security for NVM-Based Training-in-Memory Systems

Abstract

Non-volatile memory (NVM)-based training-inmemory (TIME) systems have emerged that can process the NN training in an energy-efficient manner. However, the endurance of NVM cells is disappointing, rendering concerns about the lifetime of TIME systems, because the weights of NN models always need to be updated for thousands to millions of times during training. Gradient sparsification (GS) can alleviate this problem by preserving only a small portion of the gradients to update the weights. However, conventional GS will introduce non-uniform writes on different cells across the whole NVM crossbars, which significantly reduces the excepted available lifetime. Moreover, an adversary can easily launch malicious training tasks to exactly wear-out the target cells and fast break down the system.In this paper, we propose an efficient and effective framework, referred as SGS-ARS, to improve the lifetime and security of TIME systems. The framework mainly contains a structured gradient sparsification (SGS) scheme for reducing the write frequency, and an aging-aware row swapping (ARS) scheme to make the writes uniform. Meanwhile, we show that the backpropagation mechanism allows the attacker to localize and update fixed memory locations and wear them out. Therefore, we introduce Random-ARS and Refresh techniques to thwart adversarial training attacks, preventing the systems from being fast broken in an extremely short time. Our experiments show that when TIME is programmed to train ResNet-50 on Imagenet dataset, 356× lifetime extension can be achieved without sacrificing the accuracy much or incurring much hardware overhead. Under adversarial environment, the available lifetime of TIME systems can still be improved by 84×.

A Lightweight Face Verification Based on Adaptive Cascade Network and Triplet Loss Function

“…With the continuous progress of deep learning, the research heat of face recognition algorithm is rising again. Compressed convolutional neural network can complete real-time highquality face detection on mobile platform [5]. Cascade convolutional neural network (CNN), which belongs to deep convolutional neural network (DCNN), can detect face more quickly by relying on lightweight module [6].…”

Reinforcement Learning-Based Joint Reliability and Performance Optimization for Hybrid-Cache Computing Servers

“…However, the reliability limitation (or constraint) is still the major problem for RRAM and hinders its mass application [8]. Their limited endurance, and especially their limited write cycles generally prevent their use for high-performance tasks [5], [9]. Hence, a hybrid-cache technology that uses both SRAM and NVM at the architecture level is proposed to address the shortcomings of both SRAM and NVM-based LLC [5], [10].…”