The unprecedented development of Internet of Things results in the explosion of spatiotemporal signals generated by smart edge devices, leading to a surge of interest in real‐time learning of such data. This imposes a big challenge to conventional digital hardware because of physically separated memory and processing units and the transistor scaling limit. Memristors are deemed a solution for efficient and portable deep learning. However, their ionic resistive switching incurs large programming stochasticity and energy, compromising their advantages in real‐time learning spatiotemporal signals. To address the aforementioned issues, we propose a novel hardware–software codesign. Hardware‐wise, the stochasticity in memristor programming is leveraged to produce random matrices for efficient in‐memory computing. Software‐wise, random convolutional‐pooling architectures are integrated with echo‐state networks that compute with the hardware random matrices and make real‐time learning affordable. The synergy of the hardware and software not only improves the performance over conventional echo‐state networks, that is, 90.94% and 91.67% (compared to baselines 88.33% and 62.50%), but also retains 187.79× and 93.66× improvement of energy efficiency compared to the digital alternatives on the representative Human Activity Recognition Using Smartphones (HAR) and CRICKET datasets, respectively. These advantages make random convolutional echo‐state network (RCESN) a promising solution for the future smart edge hardware.

Autonomous one‐shot on‐the‐fly learning copes with the high privacy, small dataset, and in‐stream data at the edge. Implementing such learning on digital hardware suffers from the well‐known von‐Neumann and scaling bottlenecks. The optical neural networks featuring large parallelism, low latency, and high efficiency offer a promising solution. However, ex‐situ training of conventional optical networks, where optical path configuration and deep learning model optimization are separated, incurs hardware, energy and time overheads, and defeats the advantages in edge learning. Here, we introduced a bio‐inspired material‐algorithm co‐design to construct a hydrogel‐based optical Willshaw model (HOWM), manifesting Hebbian‐rule‐based structural plasticity for simultaneous optical path configuration and deep learning model optimization thanks to the underlying opto‐chemical reactions. We first employed the HOWM as an all optical in‐sensor AI processor for one‐shot pattern classification, association and denoising. We then leveraged HOWM to function as a ternary content addressable memory (TCAM) of an optical memory augmented neural network (MANN) for one‐shot learning the Omniglot dataset. The HOWM empowered one‐shot on‐the‐fly edge learning leads to 1000× boost of energy efficiency and 10× boost of speed, which paves the way for the next‐generation autonomous, efficient, and affordable smart edge systems.

Intrinsic plasticity of neurons, such as spontaneous threshold lowering (STL) to modulate neuronal excitability, is key to spatial attention of biological neural systems. In-memory computing with emerging memristors is expected to solve the memory bottleneck of the von Neumann architecture commonly used in conventional digital computers and is deemed a promising solution to this bioinspired computing paradigm. Nonetheless, conventional memristors are incapable of implementing the STL plasticity of neurons due to their first-order dynamics. Here, a second-order memristor is experimentally demonstrated using yttria-stabilized zirconia with Ag doping (YSZ:Ag) that exhibits STL functionality. The physical origin of the second-order dynamics, i.e., the size evolution of Ag nanoclusters, is uncovered through transmission electron microscopy (TEM), which is leveraged to model the STL neuron. STL-based spatial attention in a spiking convolutional neural network (SCNN) is demonstrated, improving the accuracy of a multiobject detection task from 70% (20%) to 90% (80%) for the object within (outside) the area receiving attention. This second-order memristor with intrinsic STL dynamics paves the way for future machine intelligence, enabling high-efficiency, compact footprint, and hardware-encoded plasticity.