Approximation of activation functions for vector equalization based on recurrent neural networks

Abstract

Activation functions represent an essential element in all neural networks structures. They influence the overall behavior of neural networks decisively because of their nonlinear characteristic. Discrete-and continuous-time recurrent neural networks are a special class of neural networks. They have been shown to be able to perform vector equalization without the need for a training phase because they are Lyapunov stable under specific conditions. The activation function in this case depends on the symbol alphabet and is computationally complex to be evaluated. In addition, numerical instability can occur during the evaluation. Thus, there is a need for a computationally less complex and numerically stable evaluation. Especially for the continuous-time recurrent neural network, the evaluation must be suitable for an analog implementation. In this paper, we introduce an approximation of the activation function for vector equalization with recurrent neural networks. The activation function is approximated as a sum of shifted hyperbolic tangent functions, which can easily be realized in analog by a differential amplifier. Based on our ongoing research in this field, the analog implementation of vector equalization with recurrent neural networks is expected to improve the power/speed ratio by several order of magnitude compared with the digital one.

Advanced Low Power High Speed Nonlinear Signal Processing: An Analog VLSI Example.

“…18 Layout of the vector equalizer and pin configuration. 9,10,11,12,23,24,25,26), six pins for the weights configuration (pads 13,14,18,19,20,21), reset (pad 15), voltage supplies (pads 16,17,22) and grounds (square pads). The active area is approximately 0.09 mm 2 , with a transistor count CNT = 171 for four neurons.…”

“…for l = t equ . The above stated conditions are valid for BPSK, but can be generalized by combining the results of [10,18,19].…”

“…When written in vector form, the set of equations in (19) corresponds to Eqs. (12), if S · α = R eq · I t and β = S/(2 · V t ).…”

A Continuous-Time Recurrent Neural Network for Joint Equalization and Decoding – Analog Hardware Implementation Aspects

“…The mapping from q c to x is performed by M. Each symbol ψ represents m bits. A special class of symbol alphabets are the socalled separable symbol alphabet ψ ðsÞ [19,20].…”

“…However, this was limited to the binary phaseshift keying (BPSK) symbol alphabet ψ ={−1, +1}. This has been generalized to complex-valued symbol alphabets in [21] by combining the results of references [20,22,32] 5 . Based thereon, it has been proven that the RNN ends in a local minimum of Eq.…”