Toward Reflective Spiking Neural Networks Exploiting Memristive Devices

Abstract

The design of modern convolutional artificial neural networks (ANNs) composed of formal neurons copies the architecture of the visual cortex. Signals proceed through a hierarchy, where receptive fields become increasingly more complex and coding sparse. Nowadays, ANNs outperform humans in controlled pattern recognition tasks yet remain far behind in cognition. In part, it happens due to limited knowledge about the higher echelons of the brain hierarchy, where neurons actively generate predictions about what will happen next, i.e., the information processing jumps from reflex to reflection. In this study, we forecast that spiking neural networks (SNNs) can achieve the next qualitative leap. Reflective SNNs may take advantage of their intrinsic dynamics and mimic complex, not reflex-based, brain actions. They also enable a significant reduction in energy consumption. However, the training of SNNs is a challenging problem, strongly limiting their deployment. We then briefly overview new insights provided by the concept of a high-dimensional brain, which has been put forward to explain the potential power of single neurons in higher brain stations and deep SNN layers. Finally, we discuss the prospect of implementing neural networks in memristive systems. Such systems can densely pack on a chip 2D or 3D arrays of plastic synaptic contacts directly processing analog information. Thus, memristive devices are a good candidate for implementing in-memory and in-sensor computing. Then, memristive SNNs can diverge from the development of ANNs and build their niche, cognitive, or reflective computations.

Spatial Computing in Modular Spiking Neural Networks with a Robotic Embodiment

“…However, all these difficulties are technical and can be solved using a neurohybrid approach with memristive devices instead of internetwork axons. Memristors and memristive systems [57], which are implemented in the form of a CMOS-compatible nanostructure with a memory effect, are ideally suited for the role of such connections [44,58]. Recently, the first step in this direction has been taken: commercial memristive devices with the effect of short-term plasticity are used to arrange communication between individual subnets in vitro and provide synchronous activity of target subnets under the control of the source subnet [59].…”

“…Earlier, we formulated the basic principles of associative learning in SNNs [33,43,44]. They are (i) Hebbian learning (using STDP), (ii) synaptic competition or competition of SNN inputs, and (iii) neural competition or competition of SNN outputs.…”

Adapted MLP-Mixer network based on crossbar arrays of fast and multilevel switching (Co–Fe–B)x(LiNbO3)100−x nanocomposite memristors

“…33 Another way to mitigate the problem of memristive variations is to realize spiking NCSs with bio-inspired algorithms. [34][35][36][37][38] Although there is certain progress in the training of spiking NCSs, such as deep learning-inspired approaches, 39 surrogate gradient learning 40 and Python packages for spiking NCSs modelling like SpikingJelly 41 and SNNTorch, 39 efficient training algorithms for spiking NCSs are still underdeveloped, which complicates the transfer of memristor-based spiking NCSs from the current device level to a large system level. 42 Consequently, the search for new efficient memristor-based NCS architectures and training algorithms is of high interest.…”

Overview of Spiking Neural Network Learning Approaches and Their Computational Complexities

“…Reflective SNNs can make use of their inherent dynamics to mimic complicated, nonreflexive brain functions, such as the creation of new skills from previously learned ones. SNNs can be implemented as analog computational systems [ 13 ]. In addition to the abovementioned areas of application, SNNs are also used in the areas of cognitive processing.…”

“…As mentioned, the subject of SNNs involves many different hardware architectures and many different applications and research areas. SNNs are also the subject of many review articles [ 13 , 35 ]. In this review, we focus on the main learning approaches for SNNs in terms of their efficiency on synchronous digital hardware.…”