Nonlinear adaptive prediction of speech with a pipelined recurrent neural network

Baltersee, J.; Chambers, J.A.

doi:10.1109/78.705435

Cited by 60 publications

(45 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Notice, that the maximum value for is as shown in (13). Hence, the upper bound of an estimate of becomes (16) In our experiments, the values of were such that , which gives (17) For , which is the upper bound of the influence of the amplitude of to the amplitude of .…”

Section: The Effects Of Nestingmentioning

confidence: 99%

“…The output of the PRNN was then fed into the LMS filter in order to produce the predicted signal of the nonlinear predictor. As our aim is to improve the performance of the PRNN part, and the LMS linear predictor was shown to contribute with approximately 2 dB toward the total prediction gain [12], [13], we shall concentrate on the PRNN part of the nonlinear predictor only.…”

Section: A the Haykin-li's Nonlinear Predictormentioning

confidence: 99%

“…Now, let us provide insight into the values of the gradient for the nonlinear logistic activation function (1). The first derivative of the logistic function, which represents a gradient of the activation function in terms of the learning algorithms based upon (1) is given by (13) The maximum value of the function (13) for the common choice of is . As the nested structure from (12) actually comprises the weight matrix , the output of the PRNN, which is in effect a conditional mean predictor, can be expressed as (14) In order to measure the influence of the output of a distant module of the PRNN to its overall output (Fig.…”

Section: The Effects Of Nestingmentioning

confidence: 99%

“…Furthermore, in Section IV, the role of the influence of the forgetting factor to the gradient-based learning of the PRNN is presented, and a new solution for weighting of the modules in the PRNN is proposed. In Section V, the performance of the proposed scheme is compared to the performances of known schemes [1], [12], [13], and it is shown that the proposed scheme substantially outperforms the existing ones. Finally, in Section VI, the main results presented in the paper are summarized.…”

mentioning

confidence: 99%

“…However, the variant of this algorithm used for training the PRNN suffers from some serious drawbacks. In [12] and [13], an improved RTRL-based algorithm was presented, together with the extended Kalman filter (EKF)-based algorithm for training the PRNN. Both of them outperformed the originally proposed algorithm.…”

mentioning

confidence: 99%

See 4 more Smart Citations

Toward an optimal PRNN-based nonlinear predictor

Mandic

Chambers

1999

IEEE Trans. Neural Netw.

Self Cite

View full text Add to dashboard Cite

Abstract-We present an approach for selecting optimal parameters for the pipelined recurrent neural network (PRNN) in the paradigm of nonlinear and nonstationary signal prediction. Although there has recently been progress in terms of algorithms for training the PRNN, no account has been made of some inherent features of the PRNN. We therefore provide a study of the role of nesting, which is inherent to the PRNN architecture. The corresponding number of nested modules needed for a certain prediction task, and their contribution toward the final prediction gain (PG) give a thorough insight into the way the PRNN performs, and offers solutions for optimization of its parameters. In particular, nesting, which is a contractive function by its nature, allows the forgetting factor in the cost function of the PRNN to exceed unity, hence it becoms an emphasis factor. This compensates for the small contribution of the distant modules to the prediction process, due to nesting, and helps to circumvent the problem of vanishing gradient, experienced in RNN's for prediction. The PRNN, with its parameters chosen based upon the established criteria, is shown to outperform the linear least mean square (LMS) and recursive least squares (RLS) predictors, as well as previously proposed PRNN schemes, at no expense of additional computational complexity.

show abstract

Section: The Effects Of Nestingmentioning

confidence: 99%

Section: A the Haykin-li's Nonlinear Predictormentioning

confidence: 99%