On the convergence behavior of the LMS and the normalized LMS algorithms

Slock, Dirk

doi:10.1109/78.236504

Cited by 420 publications

(216 citation statements)

References 31 publications

Supporting

Mentioning

201

Contrasting

Unclassified

Order By: Relevance

“…Comparing (13) and the result in [10, Eq. 22], it can be found that they are identical except that all the integrals in [10] are coupled in their original forms.…”

Section: Mean Square Behaviormentioning

confidence: 99%

“…Consequently, the works in [9,10] only concentrated on certain special cases of eigenvalue distribution of the input autocorrelation matrix. In [12,13], particular or simplified input data model was introduced to facilitate the performance analysis so that useful analytical expressions can still be derived. In [14], the averaging principle was invoked to simplify the expectations involved in the difference equations.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Convergence Behavior of NLMS Algorithm for Gaussian Inputs: Solutions Using Generalized Abelian Integral Functions and Step Size Selection

Chan

Zhou

2009

J Sign Process Syst Sign Image Video Technol

View full text Add to dashboard Cite

This paper studies the mean and mean square convergence behaviors of the normalized least mean square (NLMS) algorithm with Gaussian inputs and additive white Gaussian noise. Using the Price's theorem and the framework proposed by Bershad in IEEE Transactions on Acoustics, Speech, and Signal Processing (1986, 1987), new expressions for the excess mean square error, stability bound and decoupled difference equations describing the mean and mean square convergence behaviors of the NLMS algorithm using the generalized Abelian integral functions are derived. These new expressions which closely resemble those of the LMS algorithm allow us to interpret the convergence performance of the NLMS algorithm in Gaussian environment. The theoretical analysis is in good agreement with the computer simulation results and it also gives new insight into step size selection.

show abstract

“…Comparing (13) and the result in [10, Eq. 22], it can be found that they are identical except that all the integrals in [10] are coupled in their original forms.…”

Section: Mean Square Behaviormentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Convergence Behavior of NLMS Algorithm for Gaussian Inputs: Solutions Using Generalized Abelian Integral Functions and Step Size Selection

Chan

Zhou

2009

J Sign Process Syst Sign Image Video Technol

View full text Add to dashboard Cite

show abstract

“…There has been research in the past focusing on the comparison between the LMS and the NLMS algorithms [22] - [24]. In 1993, Slock [24] studied the convergence behavior of both the algorithms and concluded that the NLMS algorithm is a potentially faster converging algorithm compared to the LMS algorithm.…”

Section: ε-Nlms Algorithmmentioning

confidence: 99%

“…In 1993, Slock [24] studied the convergence behavior of both the algorithms and concluded that the NLMS algorithm is a potentially faster converging algorithm compared to the LMS algorithm. However, faster convergence comes at a cost of high computational complexity.…”

Section: ε-Nlms Algorithmmentioning

confidence: 99%

Diffusion normalized least mean squares over wireless sensor networks

Baqi

Zerguine

Saeed

2013

2013 9th International Wireless Communications and Mobile Computing Conference (IWCMC)

View full text Add to dashboard Cite

“…However, for fixed schedules, the NLMS algorithm is known to trade off transient performance for asymptotic performance. Slock suggested an "optimal" step-size to be used with NLMS [12]. Following a heuristic analysis he concludes that NLMS performs the same as RLS as far as sensitivity to the eigenvalue spread is concerned.…”

Section: Introductionmentioning

confidence: 99%

LMS-2: Towards an algorithm that is as cheap as LMS and almost as efficient as RLS

Yao

Bhatnagar

Szepesvári

2009

Proceedings of the 48h IEEE Conference on Decision and Control (CDC) Held Jointly With 2009 28th Chinese Control Conference

View full text Add to dashboard Cite

Abstract-We consider linear prediction problems in a stochastic environment. The least mean square (LMS) algorithm is a well-known, easy to implement and computationally cheap solution to this problem. However, as it is well known, the LMS algorithm, being a stochastic gradient descent rule, may converge slowly. The recursive least squares (RLS) algorithm overcomes this problem, but its computational cost is quadratic in the problem dimension. In this paper we propose a two timescale stochastic approximation algorithm which, as far as its slower timescale is considered, behaves the same way as the RLS algorithm, while it is as cheap as the LMS algorithm. In addition, the algorithm is easy to implement. The algorithm is shown to give estimates that converge to the best possible estimate with probability one. The performance of the algorithm is tested in two examples and it is found that it may indeed offer some performance gain over the LMS algorithm.

show abstract

On the convergence behavior of the LMS and the normalized LMS algorithms

Cited by 420 publications

References 31 publications

Convergence Behavior of NLMS Algorithm for Gaussian Inputs: Solutions Using Generalized Abelian Integral Functions and Step Size Selection

Convergence Behavior of NLMS Algorithm for Gaussian Inputs: Solutions Using Generalized Abelian Integral Functions and Step Size Selection

Diffusion normalized least mean squares over wireless sensor networks

LMS-2: Towards an algorithm that is as cheap as LMS and almost as efficient as RLS

Contact Info

Product

Resources

About