Neuromorphic architectures with non-volatile memory (NVM) implement biological neurons and synapses to execute spiking neural networks (SNNs). To access synaptic weights, an NVM cell's peripheral circuit drives current through the cell using a high bias voltage, generated from an on-chip charge pump. High-voltage operations induce aging of CMOS devices in the charge pump, leading to negative bias temperature instability (NBTI) and hot carrier injection (HCI) generated defects. Therefore, charge-pump aging poses a significant threat to the operating lifetime of neuromorphic architectures. Discharging a stressed charge pump periodically can lower its aging rate, but makes the architecture unavailable to process spikes while its charge pumps are being discharged. This introduces delay in spike propagation, which impacts inter-spike interval (ISI), leading to information loss and challenging the integrity of SNNs. This performance and lifetime trade-off depends on the SNN workload being executed. In this paper, we propose a novel framework to exploit workload-specific performance and lifetime trade-offs in neuromorphic computing. Our framework first extracts the precise times at which spikes are generated on all synapses of a SNN workload. This timing information is then used wihtin a new analytical formulation to estimate aging of charge pumps based on the SNN's mapping to the hardware and the power delivery architecture of charge pumps. We use the developed framework to optimize the mapping of neurons and synapses at design time and to schedule the discharge of stressed charge pumps at run time to maximize their lifetime, without significantly hurting the workload's performance.

Backpropagation (BP) is the most successful and widely used algorithm in deep learning. However, the computations required by BP are challenging to reconcile with known neurobiology. This difficulty has stimulated interest in more biologically plausible alternatives to BP. One such algorithm is the inference learning algorithm (IL). IL has close connections to neurobiological models of cortical function and has achieved equal performance to BP on supervised learning and auto-associative tasks. In contrast to BP, however, the mathematical foundations of IL are not well-understood. Here, we develop a novel theoretical framework for IL. Our main result is that IL closely approximates an optimization method known as implicit stochastic gradient descent (implicit SGD), which is distinct from the explicit SGD implemented by BP. Our results further show how the standard implementation of IL can be altered to better approximate implicit SGD. Our novel implementation considerably improves the stability of IL across learning rates, which is consistent with our theory, as a key property of implicit SGD is its stability. We provide extensive simulation results that further support our theoretical interpretations and also demonstrate IL achieves quicker convergence when trained with small mini-batches while matching the performance of BP for large mini-batches.

No abstract

No abstract

No abstract