Tracking performance of leakage LMS for chirped signals

Ting, Lok-Kee; Cowan, C.F.N.; Woods, Roger; Cork, P. R.; Sprigings, C.J.

doi:10.1109/sips.2001.957335

Cited by 4 publications

(3 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This inequality confirms that the momentum algorithm can achieve a faster rate in deterministic optimization and, moreover, this faster rate cannot be attained by standard gradient descent. Motivated by these useful acceleration properties in the deterministic context, momentum terms have been subsequently introduced into stochastic optimization algorithms as well (Polyak, 1987;Proakis, 1974;Sharma et al, 1998;Shynk and Roy, June 1988;Roy and Shynk, 1990;Tugay and Tanik, 1989;Bellanger, 2001;Wiegerinck et al, 1994;Hu et al, 2009;Xiao, 2010;Lan, 2012;Ghadimi and Lan, 2012;Zhong and Kwok, 2014) and applied, for example, to problems involving the tracking of chirped sinusoidal signals (Ting et al, 2000) or deep learning (Sutskever et al, 2013;Kahou et al, 2013;Szegedy et al, 2015;Zareba et al, 2015). However, the analysis in this paper will show that the advantages of the momentum technique for deterministic optimization do not necessarily carry over to the adaptive online setting due to the presence of stochastic gradient noise (which is the difference between the actual gradient vector and its approximation).…”

Section: Acceleration Methodsmentioning

confidence: 99%

On the influence of momentum acceleration on online learning

Yuan

Ying

Sayed

2016

2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

The article examines in some detail the convergence rate and mean-square-error performance of momentum stochastic gradient methods in the constant step-size and slow adaptation regime. The results establish that momentum methods are equivalent to the standard stochastic gradient method with a re-scaled (larger) step-size value. The size of the re-scaling is determined by the value of the momentum parameter. The equivalence result is established for all time instants and not only in steady-state. The analysis is carried out for general strongly convex and smooth risk functions, and is not limited to quadratic risks. One notable conclusion is that the well-known benefits of momentum constructions for deterministic optimization problems do not necessarily carry over to the adaptive online setting when small constant step-sizes are used to enable continuous adaptation and learning in the presence of persistent gradient noise. From simulations, the equivalence between momentum and standard stochastic gradient methods is also observed for non-differentiable and non-convex problems.

show abstract

Section: Acceleration Methodsmentioning

confidence: 99%

On the influence of momentum acceleration on online learning

Yuan

Ying

Sayed

2016

2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…This is why the improvement percentage of MSE is more significant for smaller step sizes. Figure 6 shows the MSE comparison between the proposed Markov-chain based booster and momentum LMS [25]. The proposed booster uses the Markov chain con- …”

Section: Resultsmentioning

confidence: 99%

A Novel Convergence Accelerator for the LMS Adaptive Filter

Sheu

Woo

Wen

2011

Circuits Syst Signal Process

View full text Add to dashboard Cite

Due to its ease of implementation, the least mean square (LMS) algorithm is one of the most well-known algorithms for mobile communication systems. However, the main limitation of this approach is its relatively slow convergence rate. This paper proposes a booster using the Markov chain concept to speed up the convergence rate of LMS algorithms. The nature of Markov chains makes it possible to exploit past information in the updating process. According to the central limit theorem, the transition matrix has a smaller variance than that of the weight itself. As a result, the weight transition matrix converges faster than the weight itself. Therefore, the proposed Markov-chain based booster is able to track variations in signal characteristics and simultaneously accelerate the rate of convergence for LMS algorithms. Simulation results show that the Markov-chain based booster allows an LMS algorithm to effectively increase the convergence rate and further approach the Wiener solution. This approach also markedly reduces the mean square error while improving the convergence rate.

show abstract

“…In practical applications, in order to ensure the stability of the filter, the value of the leakage factor χ is generally 0.95 < χ < 1 [37]. In this section, it is set to χ = 0.97, and the range of the adaptive step size is 0 < μ < 1.97 obtained by (13).…”

Section: B Parameters Selectionmentioning

confidence: 99%

Adaptive Clutter Suppression and Detection Algorithm for Radar Maneuvering Target With High-Order Motions Via Sparse Fractional Ambiguity Function

Chen

Yu²,

Huang

et al. 2020

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Radar maneuvering target detection in clutter background should not only consider the complex characteristics of the target to accumulate its energy as much as possible, but also suppress clutter to improve the signal-to-clutter ratio (SCR). The traditional fractional domain transform-based detection method requires parameters match searching, which costs heavy computational burden in case of a large amount of data. Sparse FT and sparse fractional FT can obtain high-resolution sparse representation of the target, but the signal sparsity needs to be known before, and the sparse representation performance is poor in clutter background. In this article, adaptive filtering method is introduced into the sparse fractional ambiguity function (SFRAF) method, and a SFRAF domain adaptive clutter suppression and highly maneuvering target detection algorithm is proposed, which is named as adaptive SFRAF (ASFRAF). The ASFRAF domain iterative filtering operation can suppress the clutter while retaining the signal energy as much as possible. Simulation results and measured radar data processing results show that the proposed algorithm can overcome the limitation of the SFRAF on the sparsity preset value and achieve high efficiency and robust detection of high-order phase maneuvering targets under a low SCR environment.

show abstract

Tracking performance of leakage LMS for chirped signals

Cited by 4 publications

References 6 publications

On the influence of momentum acceleration on online learning

On the influence of momentum acceleration on online learning

A Novel Convergence Accelerator for the LMS Adaptive Filter

Adaptive Clutter Suppression and Detection Algorithm for Radar Maneuvering Target With High-Order Motions Via Sparse Fractional Ambiguity Function

Contact Info

Product

Resources

About