An On-Chip Learning Neuromorphic Autoencoder With Current-Mode Transposable Memory Read and Virtual Lookup Table

Cho, Hwasuk; Son, Hyeonwi; Seong, Kihwan; Kim, Byungsub; Park, Hong June; Sim, Jae Yoon

doi:10.1109/tbcas.2017.2762002

Cited by 18 publications

(3 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To support this bidirectional memory access, memory cells for the synaptic weights must be transposable or accessed row-by-row successively. Transposable memory [22] cells and array structures cause area overhead owing to the additional transistors and metal lines required. In addition, frequent memory access causes an increase in energy for learning a single image.…”

Section: B Post-neuron Spike-referred Stdp (Pr-stdp)mentioning

confidence: 99%

Area- and Energy-Efficient STDP Learning Algorithm for Spiking Neural Network SoC

Kim

Choi

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Recently, spiking neural networks have gained attention owing to their energy efficiency. Allto-all spike-time dependent plasticity is a popular learning algorithm for spiking neural networks because it is suitable for nondifferentiable spike event-based learning and requires fewer computations than backpropagation-based algorithms. However, the hardware implementation of all-to-all spike-time dependent plasticity is limited by the large storage area required for spike history and large energy consumption caused by frequent memory access. We propose a time-step scaled spike-time dependent plasticity to reduce the storage area required for spike history by reducing the area of the spike-time dependent plasticity learning circuit by 60% and a post-neuron spike-referred spike-time dependent plasticity to reduce the energy consumption by 99.1% by efficiently accessing the memory while learning. The accuracy of Modified National Institute of Standards and Technology image classification degraded by less than 2% when both time-step scaled spike-time dependent plasticity and post-neuron spike-referred spike-time dependent plasticity were applied. Thus, the proposed hardware-friendly spike-time dependent plasticity algorithms make all-to-all spike-time dependent plasticity implementable in more compact areas while reducing energy consumption and experiencing insignificant accuracy degradation. INDEX TERMS Spike-time dependent plasticity (STDP), Time-step scaled STDP (TS-STDP), Postneuron spike-referred STDP (PR-STDP), Spiking neural network (SNN) Wij Learning neuron Synaptic weight Off-chip On-chip Pre-neuron spike Post-neuron spike ... ... FIGURE 1. Concept of SNN and its structure.

show abstract

Section: B Post-neuron Spike-referred Stdp (Pr-stdp)mentioning

confidence: 99%

Area- and Energy-Efficient STDP Learning Algorithm for Spiking Neural Network SoC

Kim

Choi

et al. 2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…There are two methods for training the weight values: onchip and off-chip trainings, and both methods require memristor devices that can be programmed to optimized weight state. In the on-chip training, which is called in situ training, the weight of the synaptic device is updated in the chip, so additional peripheral circuits and better memristor endurance characteristics are needed to apply corresponding teaching signals and bear a number of learning events [32][33][34]. In contrast, in the off-chip training, which is called ex situ training, pre-trained weight values by software algorithms are transferred to individual cells in the synaptic array [35][36][37]; therefore, a memristor with programmable multilevel conductance state is required with accurate tuning method to precisely realize floating-point weight values, but there would be inevitable tuning errors compared with pre-trained weight values [38,39].…”

Section: Introductionmentioning

confidence: 99%

Intrinsic variation effect in memristive neural network with weight quantization

Park¹,

Song²,

Youn³

et al. 2022

Nanotechnology

View full text Add to dashboard Cite

To analyze the effect of the intrinsic variations of the memristor device on the neuromorphic system, we fabricated 32×32 Al2O3/TiOx-based memristor crossbar array and implemented 3-bit multilevel conductance as weight quantization by utilizing the switching characteristics to minimize the performance degradation of the neural network. The tuning operation of 8 weight levels was confirmed with a tolerance of ±4 μA (±40 μS). The endurance and retention characteristics were also verified, and the random telegraph noise (RTN) characteristics were measured according to the weight range to evaluate the internal stochastic variation effect. Subsequently, a memristive neural network was constructed by off-chip training with differential memristor pairs for the modified National Institute of Standards and Technology (MNIST) handwritten dataset. The pre-trained weights were quantized, and the classification accuracy was evaluated by applying the intrinsic variations to each quantized weight. The intrinsic variations were applied using the measured weight inaccuracy given by the tuning tolerance, RTN characteristics, and the fault device yield. These results should be considered when the pre-trained weights are transferred to a memristive neural network during the off-chip training.

show abstract

“…Some DNNs, such as some MLPs [ 24 , 25 ], RBMs [ 26 , 27 ], and CNNs [ 28 – 31 ], have been developed as dedicated chips. One of the report uses RBMs for training and AEs for inference [ 32 ]. In addition to these examples of the hardware implementations of DNNs, FPGA implementations of CNNs and RBMs have also been reported in [ 33 – 38 ].…”

Section: Introductionmentioning

confidence: 99%

A shared synapse architecture for efficient FPGA implementation of autoencoders

2018

View full text Add to dashboard Cite

This paper proposes a shared synapse architecture for autoencoders (AEs), and implements an AE with the proposed architecture as a digital circuit on a field-programmable gate array (FPGA). In the proposed architecture, the values of the synapse weights are shared between the synapses of an input and a hidden layer, and between the synapses of a hidden and an output layer. This architecture utilizes less of the limited resources of an FPGA than an architecture which does not share the synapse weights, and reduces the amount of synapse modules used by half. For the proposed circuit to be implemented into various types of AEs, it utilizes three kinds of parameters; one to change the number of layers’ units, one to change the bit width of an internal value, and a learning rate. By altering a network configuration using these parameters, the proposed architecture can be used to construct a stacked AE. The proposed circuits are logically synthesized, and the number of their resources is determined. Our experimental results show that single and stacked AE circuits utilizing the proposed shared synapse architecture operate as regular AEs and as regular stacked AEs. The scalability of the proposed circuit and the relationship between the bit widths and the learning results are also determined. The clock cycles of the proposed circuits are formulated, and this formula is used to estimate the theoretical performance of the circuit when the circuit is used to construct arbitrary networks.

show abstract

An On-Chip Learning Neuromorphic Autoencoder With Current-Mode Transposable Memory Read and Virtual Lookup Table

Cited by 18 publications

References 22 publications

Area- and Energy-Efficient STDP Learning Algorithm for Spiking Neural Network SoC

Area- and Energy-Efficient STDP Learning Algorithm for Spiking Neural Network SoC

Intrinsic variation effect in memristive neural network with weight quantization

A shared synapse architecture for efficient FPGA implementation of autoencoders

Contact Info

Product

Resources

About