Convolutional Echo‐State Network with Random Memristors for Spatiotemporal Signal Classification

Abstract

The unprecedented development of Internet of Things results in the explosion of spatiotemporal signals generated by smart edge devices, leading to a surge of interest in real‐time learning of such data. This imposes a big challenge to conventional digital hardware because of physically separated memory and processing units and the transistor scaling limit. Memristors are deemed a solution for efficient and portable deep learning. However, their ionic resistive switching incurs large programming stochasticity and energy, compromising their advantages in real‐time learning spatiotemporal signals. To address the aforementioned issues, we propose a novel hardware–software codesign. Hardware‐wise, the stochasticity in memristor programming is leveraged to produce random matrices for efficient in‐memory computing. Software‐wise, random convolutional‐pooling architectures are integrated with echo‐state networks that compute with the hardware random matrices and make real‐time learning affordable. The synergy of the hardware and software not only improves the performance over conventional echo‐state networks, that is, 90.94% and 91.67% (compared to baselines 88.33% and 62.50%), but also retains 187.79× and 93.66× improvement of energy efficiency compared to the digital alternatives on the representative Human Activity Recognition Using Smartphones (HAR) and CRICKET datasets, respectively. These advantages make random convolutional echo‐state network (RCESN) a promising solution for the future smart edge hardware.

Thin-film transistor for temporal self-adaptive reservoir computing with closed-loop architecture

“…S18. It is reported that completely random convolution ahead of RC can effectively promote accuracy although having a slight drop compared to the fully trained convolutional kernel ( 52 ). Then, we convert the convolution outcome into the pulse sequence with the integrate-and-fire principle, which will be inputted into the hardware reservoir.…”

Recent Advances and Future Prospects for Memristive Materials, Devices, and Systems

“…213 In addition, pairing echo state network-based temporal feature extractors with random convolutional-pooling architecture-based spatial feature extractors can naturally learn spatial−temporal signals at low cost. 214 Bayesian inference may also exploit the programming variation of the resistive memory. Hardware-wise, the 1transistor-1-resistive memory (1T1R) array allows physically implementation of the proposal distribution using the cycle-tocycle and device-to-device programming variation of resistive memory.…”

“…Notably, such an echo state graph neural network can serve as the graph feature extractor when combined with a trainable projection layer and associative memory, forming a memory-augmented graph neural network for few-shot graph learning . In addition, pairing echo state network-based temporal feature extractors with random convolutional-pooling architecture-based spatial feature extractors can naturally learn spatial–temporal signals at low cost …”

A 65 nm 12.92-nJ/Inference Mixed-Signal Neuromorphic Processor for Image Classification

“…Memristor array is a promising solution to implement the reservoir, as their weights exhibit stochasticity inherently after programming [11]. This process can be stabilized using additional techniques such as injecting a delayed pulse [12].…”