Long Short-Term Memory Network Design for Analog Computing

Abstract

We present an analog-integrated circuit implementation of long short-term memory network, which is compatible with digital CMOS technology. We have used multiple-input floating gate MOSFETs as both the front-end to obtain converted analog signals and the differential pairs in proposed analog multipliers. Analog crossbar is built by the analog multiplier processing matrix and bitwise multiplications. We have shown that using current signals as internal transmission signals can largely reduce computation delay, compared to the digital implementation. We also have introduced analog blocks to work as activation functions for the algorithm. In the back-end of our design, we have used current comparators to achieve the output to be readable to external digital systems. We have designed the LSTM network with the matrix size of 16 × 16 in TSMC 180nm CMOS technology. The post-layout simulations show that the latency of one computing cycle is 1.19ns without memory, and power dissipation of the single analog LSTM computing core with 2 kilobytes SRAM at 200MHz is 460.3mW. The overhead of power dissipation due to SRAM access is 8.3%, in which the computing of each LSTM layer requires one computing cycle. The energy efficiency is 0.95TOP/s/W.

Recurrent Neural Networks: An Embedded Computing Perspective

“…Analog computing and processing in memory: Analog computing [98] and processing in memory [71], [101] have shown promising performance, especially in energy efficiency. Analog crossbar based matrix to vector multiplication units can provide low latency and computing in memory overcomes the memory access problems.…”

“…• Analog computing Analog computing is a good candidate for neural network accelerators [98]. Analog neural networks [99] and analog CNNs [100] have been studied recently.…”

“…Analog neural networks [99] and analog CNNs [100] have been studied recently. Interestingly, RNN implementations using analog computing have started to attract attention from researchers [98], [101]. Analog computing brings significant benefits, especially for the critical matrix-vector computation, by making it both faster and more energyefficient.…”

“…It can be observed that the highest energy efficiency was achieved by A7 [120] and A3 [98]. Both implementations used analog crossbar based matrix to vector multiplications.…”

Anomaly Detection for Controller Area Networks Using Long Short-Term Memory

“…Ensembles of LSTM networks have been used in behavior recognition [36]. Attempts have been made to implement LSTMs in hardware and these have the potential of speeding up offline LSTM training [37].…”