A 55nm time-domain mixed-signal neuromorphic accelerator with stochastic synapses and embedded reinforcement learning for autonomous micro-robots

Amravati, Anvesha; Nasir, Saad Bin; Sivaram, Thangadurai; Yoon, Insik; Raychowdhury, Arijit

doi:10.1109/isscc.2018.8310215

Cited by 59 publications

(31 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Compared to the work of Bong et al (2017), the boosting of energy efficiency of our design is 2 ×, corresponding to D LSTM =1. Design in the work of Amravati et al (2018) is suitable for applications in portable devices, and its energy efficiency is higher than in our work. In this work, digital-to-pulse converter (DPC) is used as signal interface.…”

Section: Memory Accessmentioning

confidence: 78%

See 1 more Smart Citation

Long Short-Term Memory Network Design for Analog Computing

Zhao

Srivastava

Peng

et al. 2019

J. Emerg. Technol. Comput. Syst.

View full text Add to dashboard Cite

We present an analog-integrated circuit implementation of long short-term memory network, which is compatible with digital CMOS technology. We have used multiple-input floating gate MOSFETs as both the front-end to obtain converted analog signals and the differential pairs in proposed analog multipliers. Analog crossbar is built by the analog multiplier processing matrix and bitwise multiplications. We have shown that using current signals as internal transmission signals can largely reduce computation delay, compared to the digital implementation. We also have introduced analog blocks to work as activation functions for the algorithm. In the back-end of our design, we have used current comparators to achieve the output to be readable to external digital systems. We have designed the LSTM network with the matrix size of 16 × 16 in TSMC 180nm CMOS technology. The post-layout simulations show that the latency of one computing cycle is 1.19ns without memory, and power dissipation of the single analog LSTM computing core with 2 kilobytes SRAM at 200MHz is 460.3mW. The overhead of power dissipation due to SRAM access is 8.3%, in which the computing of each LSTM layer requires one computing cycle. The energy efficiency is 0.95TOP/s/W.

show abstract

Section: Memory Accessmentioning

confidence: 78%

“…In Table 4, we also compare the proposed design with other work of ASP based on neural network (Bong et al 2017;Amravati et al 2018). In the work of Bong et al (2017), ADC array is designed in another chip considering limited power budget in a single chip.…”

Section: Memory Accessmentioning

confidence: 99%

Long Short-Term Memory Network Design for Analog Computing

Zhao

Srivastava

Peng

et al. 2019

J. Emerg. Technol. Comput. Syst.

View full text Add to dashboard Cite

show abstract

“…Computationally, this is motivated by biology where memory and compute are interleaved and global movement of data is minimal. New neuromorphic architectures for deep learning (Shin et al, 2017; Lee et al, 2018, 2019) and reinforcement learning (Amaravati et al, 2018a,b; Cao et al, 2019; Kim et al, 2019) seek to apply this constraint to avoid the communication overhead. However, as we have noted earlier, BP violates this constraint.…”

Section: Resultsmentioning

confidence: 99%

Direct Feedback Alignment With Sparse Connections for Local Learning

Crafton¹,

Parihar²,

Gebhardt³

et al. 2019

Front. Neurosci.

Self Cite

View full text Add to dashboard Cite

Recent advances in deep neural networks (DNNs) owe their success to training algorithms that use backpropagation and gradient-descent. Backpropagation, while highly effective on von Neumann architectures, becomes inefficient when scaling to large networks. Commonly referred to as the weight transport problem, each neuron's dependence on the weights and errors located deeper in the network require exhaustive data movement which presents a key problem in enhancing the performance and energy-efficiency of machine-learning hardware. In this work, we propose a bio-plausible alternative to backpropagation drawing from advances in feedback alignment algorithms in which the error computation at a single synapse reduces to the product of three scalar values. Using a sparse feedback matrix, we show that a neuron needs only a fraction of the information previously used by the feedback alignment algorithms. Consequently, memory and compute can be partitioned and distributed whichever way produces the most efficient forward pass so long as a single error can be delivered to each neuron. We evaluate our algorithm using standard datasets, including ImageNet, to address the concern of scaling to challenging problems. Our results show orders of magnitude improvement in data movement and 2× improvement in multiply-and-accumulate operations over backpropagation. Like previous work, we observe that any variant of feedback alignment suffers significant losses in classification accuracy on deep convolutional neural networks. By transferring trained convolutional layers and training the fully connected layers using direct feedback alignment, we demonstrate that direct feedback alignment can obtain results competitive with backpropagation. Furthermore, we observe that using an extremely sparse feedback matrix, rather than a dense one, results in a small accuracy drop while yielding hardware advantages. All the code and results are available under https://github.com/bcrafton/ssdfa .

show abstract

“…The simulation projects the energy consumption of the ferroelectric neuron at 45-nm technology node where an improvement of 390x is observed compared to the experimental result [15]. This projected energy consumption of the ferroelectric spiking neuron is lower than that of the analog CMOS counterparts [16], [17]. The jump in energy consumption when the output firing frequency is around 40 Hz is due to the fact that excitatory pulse input with higher frequency, after travels through the leaky integrator, increases the voltage at the gate of the MOSFET (V GM ); as a result, more current is drawn by the bottom MOSFET and energy consumption increases.…”

Section: A Excitation Of Ferroelectric Spiking Neuronsmentioning

confidence: 90%

“…The inhibition function of the ferroelectric neurons regulates the activation of excitatory neurons, increases sparsity in spiking, and enables high accuracy in unsupervised learning. In addition, the ferroelectric spiking neuron has a compact 1T-1FEFET structure, which outperforms the CMOS counterparts in terms of area [16], [17]. Moreover, with the builtin excitatory and inhibitory functions, the spiking neural network based on ferroelectric neurons only requires positive synaptic weights, which further simplifies the complexity of the spiking neural network.…”

Section: Computing With Ferroelectric Spiking Neuronmentioning

confidence: 99%

Ferroelectric Relaxation Oscillators and Spiking Neurons

Wang

Khan

2019

IEEE J. Explor. Solid-State Comput. Devices Circuits

View full text Add to dashboard Cite

We report the experimental demonstration of the ferroelectric field-effect transistor (FEFET)based relaxation oscillators and spiking neurons. The ferroelectric relaxation oscillators and the ferroelectric spiking neurons, harnessing the abrupt hysteretic transition feature of ferroelectrics, have a compact 1T-1FEFET structure. The bias conditions of the FEFET can dynamically tune the hysteresis; therefore, the dynamics of oscillations and spikings can be controlled, which enable both excitation and inhibition functions in ferroelectric spiking neurons. Such FEFET-based systems are basic building blocks for efficient computational platforms for non-von Neumann and neuromorphic computing paradigms, such as coupled oscillator networks and spiking neural networks. INDEX TERMS Ferroelectric field-effect transistor (FEFET), ferroelectrics, neuromorphic, relaxation oscillator, spiking neuron.

show abstract

A 55nm time-domain mixed-signal neuromorphic accelerator with stochastic synapses and embedded reinforcement learning for autonomous micro-robots

Cited by 59 publications

References 3 publications

Long Short-Term Memory Network Design for Analog Computing

Long Short-Term Memory Network Design for Analog Computing

Direct Feedback Alignment With Sparse Connections for Local Learning

Ferroelectric Relaxation Oscillators and Spiking Neurons

Contact Info

Product

Resources

About