Non-Linear Adaptive Prediction of Speech with a Pipelined Recurrent Neural Network and Advanced Learning Algorithms

Mandic, Danilo P.; Baltersee, J.; Chambers, J.A.

doi:10.1007/978-1-4612-1768-8_21

Cited by 10 publications

(20 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The output of the PRNN was then fed into the LMS filter in order to produce the predicted signal of the nonlinear predictor. As our aim is to improve the performance of the PRNN part, and the LMS linear predictor was shown to contribute with approximately 2 dB toward the total prediction gain [12], [13], we shall concentrate on the PRNN part of the nonlinear predictor only.…”

Section: A the Haykin-li's Nonlinear Predictormentioning

confidence: 99%

Section: The Effects Of Nestingmentioning

confidence: 99%

“…That being the case, the functional dependence of the output of the network can be expressed as (12) where it was assumed that all the neurons in the network operate with the same activation function , and for the sake of simplicity, the functional dependence of the weight matrix to the nested nonlinearities was omitted. The result given in (12) gives the PRNN its enhanced computing power compared to the conventional RNN. Indeed, it is a universal approximator in the sense that a PRNN with appropriate training can approximate any nonlinear autoregressive moving average (NARMA) process to any desired degree of accuracy, provided that a sufficient number of hidden neurons is available [15], [16].…”

Section: The Effects Of Nestingmentioning

confidence: 99%

“…However, the variant of this algorithm used for training the PRNN suffers from some serious drawbacks. In [12] and [13], an improved RTRL-based algorithm was presented, together with the extended Kalman filter (EKF)-based algorithm for training the PRNN. Both of them outperformed the originally proposed algorithm.…”

mentioning

confidence: 99%

“…Furthermore, in Section IV, the role of the influence of the forgetting factor to the gradient-based learning of the PRNN is presented, and a new solution for weighting of the modules in the PRNN is proposed. In Section V, the performance of the proposed scheme is compared to the performances of known schemes [1], [12], [13], and it is shown that the proposed scheme substantially outperforms the existing ones. Finally, in Section VI, the main results presented in the paper are summarized.…”

mentioning

confidence: 99%

See 4 more Smart Citations

Toward an optimal PRNN-based nonlinear predictor

Mandic

Chambers

1999

IEEE Trans. Neural Netw.

View full text Add to dashboard Cite

Abstract-We present an approach for selecting optimal parameters for the pipelined recurrent neural network (PRNN) in the paradigm of nonlinear and nonstationary signal prediction. Although there has recently been progress in terms of algorithms for training the PRNN, no account has been made of some inherent features of the PRNN. We therefore provide a study of the role of nesting, which is inherent to the PRNN architecture. The corresponding number of nested modules needed for a certain prediction task, and their contribution toward the final prediction gain (PG) give a thorough insight into the way the PRNN performs, and offers solutions for optimization of its parameters. In particular, nesting, which is a contractive function by its nature, allows the forgetting factor in the cost function of the PRNN to exceed unity, hence it becoms an emphasis factor. This compensates for the small contribution of the distant modules to the prediction process, due to nesting, and helps to circumvent the problem of vanishing gradient, experienced in RNN's for prediction. The PRNN, with its parameters chosen based upon the established criteria, is shown to outperform the linear least mean square (LMS) and recursive least squares (RLS) predictors, as well as previously proposed PRNN schemes, at no expense of additional computational complexity.

show abstract

Section: A the Haykin-li's Nonlinear Predictormentioning

confidence: 99%

Section: The Effects Of Nestingmentioning

confidence: 99%