An Extension of the Back-Propagation Algorithm to Complex Numbers

Nitta, Tohru

doi:10.1016/s0893-6080(97)00036-1

Cited by 284 publications

(169 citation statements)

References 5 publications

Supporting

Mentioning

154

Contrasting

Unclassified

Order By: Relevance

“…This problem has been simulated with a 1-3-1 complex-value network in [12,15]. In this paper the new complex-valued WNN and conventional CVBPNN and CVWNN are applied to resolve the similar XOR problem.…”

Section: Simulation Resultsmentioning

confidence: 99%

“…According to the steepest descent rule, the weights can be updated as (11) (12) where is learning step, which is a positive constant. Combining (11) and (12), we can have (13) Finally，according to the error function formula, we can get (14) The weight update equations can be summarized by defining a quantity (15) When the active function (AF) is sigmoid function, such as (16) which is one of the most widely used AF for the artificial neural network. The first-order differential of the AF is…”

Section: The Forward Propagation Processmentioning

confidence: 99%

See 1 more Smart Citation

The New Complex-Valued Wavelet Neural Network

Jiang

2014

TELKOMNIKA

View full text Add to dashboard Cite

A new complex-valued wavelet neural network is proposed in this paper, by introducing a modified complex-valued back propagation algorithm, in which a new error function is to be minimized by the algorithm. The improvement performance is further confirmed by the simulation results, which showthat the modified algorithm is simpler than the conventional algorithm, and has better convergence, better stability and faster running speed. Keywords: complex-valued wavelet neural network (CVWNN); complex-valued back propagation (CVBP) algorithm; XOR IntroductionWavelet analysis theory is considered to be a breakthrough in the Fourier analysis and has been applied in many research areas. Wavelet transform can effectively extract the local information of the signal by scaling and translation to analyze the signal [1]. Combining the wavelets with the artificial neural network (ANN), the wavelet neural network (WNN) has been developed [2]- [4]. The ANN has many important properties such as learning, generalization, and parallel computation, although it need a large number of neurons in hidden layer and cannot converge quickly. WNN has inherited the good properties of the ANN. Moreover it can converge quickly and give high precision with reduced network size because of the time-frequency localization properties of wavelets [5].There are two types of WNN structure. The first WNN is pre-wavelet neural network, and the architecture is shown in Figure 1. The network firstly process the input signal using the orthogonal wavelet matrix, then the network put into learning and discriminating. The second WNN is called embedded wavelet neural network, the architecture of which is shown in Figure 2, in which the wavelet transform algorithm is integrated into the feed-forward neural network. In embedded wavelet neural network, wavelet functions are used in the hidden layer of the network as activation functions instead of local functions in time such as Gaussian and sigmoid functions.Li et al. [6] proposed complex-valued wavelet artificial network (CVWNN) using Haar wavelet as the hidden layer activation function (AF) in complex-valued artificial neural network (CVANN). The complex-valued wavelet neural network is the complex version of the real-valued wavelet neural network, which has complex inputs, outputs, connection weights, dilation and translation parameters, but the nonlinearity of the hidden nodes remains a real-valued function (real-valued wavelet function). CVWNN has expanded its applications in fields dealing with complex numbers such as biomedical image processing The core algorithm of the CVWNN is complex-valued BP algorithm, which is based on gradient descent often suffers from a local minima problem and has slow convergence. Many methods [12], [13] have been proposed to improve the performance, such as the convergence and the local stability. These methods usually applied adaptive activation function and added a term to the conventional error function to speed up the convergence and prevent the learning from stic...

show abstract

Section: Simulation Resultsmentioning

confidence: 99%

Section: The Forward Propagation Processmentioning

confidence: 99%

The New Complex-Valued Wavelet Neural Network

Jiang

2014

TELKOMNIKA

View full text Add to dashboard Cite

show abstract

“…Simulation results on the learning dynamics of the three-layered real-valued and complex-valued neural networks in the neighborhood of singularities support the analytical results. It should be noted here that it has been reported that the learning speed of the Complex-BP is two or three times faster than that of the Real-BP on average via computer simulations [5,16]. This is due to the learning structure of the complex-valued neural networks described above.…”

Section: Introductionmentioning

confidence: 81%

“…The complex-valued neural network can represent more information than the real-valued neural network because the inputs, the weights, the threshold values, and the outputs are all complex numbers, and the complex-valued neural network has some inherent properties such as the ability to transform geometric figures [3][4][5] and the orthogonal decision boundary [6,7].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning Dynamics of the Complex-Valued Neural Network in the Neighborhood of Singular Points

Nitta¹

2014

JCC

Self Cite

View full text Add to dashboard Cite

In this paper, the singularity and its effect on learning dynamics in the complex-valued neural network are elucidated. It has learned that the linear combination structure in the updating rule of the complex-valued neural network increases the speed of moving away from the singular points, and the complex-valued neural network cannot be easily influenced by the singular points, whereas the learning of the usual real-valued neural network can be attracted in the neighborhood of singular points, which causes a standstill in learning. Simulation results on the learning dynamics of the three-layered real-valued and complex-valued neural networks in the neighborhood of singularities support the analytical results.

show abstract