Brain-Inspired Computing: Models and Architectures

Abstract

With an exponential increase in the amount of data collected per day, the fields of artificial intelligence and machine learning continue to progress at a rapid pace with respect to algorithms, models, applications, and hardware. In particular, deep neural networks have revolutionized these fields by providing unprecedented human-like performance in solving many real-world problems such as image or speech recognition. There is also significant research aimed at unraveling the principles of computation in large biological neural networks and, in particular, biologically plausible spiking neural networks. This paper presents an overview of the brain-inspired computing models starting with the development of the perceptron and multi-layer perceptron followed by convolutional neural networks (CNNs) and recurrent neural networks (RNNs). The paper also briefly reviews other neural network models such as Hopfield neural networks and Boltzmann machines. Other models such as spiking neural networks (SNNs) and hyperdimensional computing are then briefly reviewed. Recent advances in these neural networks and graph related neural networks are then described.

A Comprehensive Survey on Nanophotonic Neural Networks: Architectures, Training Methods, Optimization, and Activations Functions

“…The philosophy behind the use of photonic circuits [ 22 , 44 , 45 ] in nano-arrangements [ 46 , 47 , 48 ] and materials is based on the need for significant improvement of the speed of transmitting and processing data and for an improvement of the energy efficiency of devices. The PNN materializations based on the aforementioned materials, which are present to this day, are classified into two main categories: with memory (stateful) and without memory (stateless), as they are concisely presented in Figure 1 [ 18 , 21 , 49 ]:…”

“…The zReLU is the rectified linear unit function, with which the positive part of its definition is received as follows [ 10 , 45 , 49 ]: …”

A Comprehensive Survey on Nanophotonics Neural Networks

“…The philosophy behind the use of photonic circuits [22], [44], [45] in nano-arrangements [46]- [48] and materials is based on the need for significant improvement of the speed of transmitting and processing data and for an improvement of the energy efficiency of devices. The PNN materializations based on the fore mentioned materials, that have been present to this day, are classified into two main categories: with memory (Stateful) and without memory (Stateless), as they are concisely presented in figure 1 below [18], [21], [49]: Moreover, the PNNs are classified according to design (waveguides or free-space optic) and according to optical training ability (Trainable) or reference only (Inference).…”

“…The zReLU is the Rectified Linear Units Function, with which the positive part of its definition is received as follows [10], [45], [49]:…”

Self-Evolutionary Neuron Model for Fast-Response Spiking Neural Networks

“…SNNs can simulate arbitrary feed-forward sigmoidal ANNs [15] and have been proven to be computationally more efficient than the neurons with sigmoidal activation function. Modern SNNs have been extensively studied in structural and neuron model design [16], [17], learning algorithms [18]- [21], information coding [22], etc., and also have found a wide range of applications, such as object recognition [23], [24], image classification [13], and series data process [25]- [27].…”