Quasi‐Newton Methods

Yuan, Ya-xiang

doi:10.1002/9780470400531.eorms0698

Cited by 4 publications

(3 citation statements)

References 62 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The L-BFGS method is a quasi-Newton method (Yuan, 2011) which achieves a similar convergence rate as Newton's method near the optimal solution. L-BFGS is widely used in practice.…”

Section: L-bfgs Powerball Methodsmentioning

confidence: 99%

On the powerball method

Yuan

Tomlin

2017

2017 29th Chinese Control and Decision Conference (CCDC)

View full text Add to dashboard Cite

We propose a new method to accelerate the convergence of optimization algorithms. This method simply adds a power coefficient γ ∈ [0, 1) to the gradient during optimization. We call this the Powerball method and analyze the convergence rate for the Powerball method for strongly convex functions. While theoretically the Powerball method is guaranteed to have a linear convergence rate in the same order of the gradient method, we show that empirically it significantly outperforms the gradient descent and Newton's method, especially during the initial iterations. We demonstrate that the Powerball method provides a 10-fold speedup of the convergence of both gradient descent and L-BFGS on multiple real datasets.

show abstract

“…The L-BFGS method is a quasi-Newton method (Yuan, 2011) which achieves a similar convergence rate as Newton's method near the optimal solution. L-BFGS is widely used in practice.…”

Section: L-bfgs Powerball Methodsmentioning

confidence: 99%

On the powerball method

Yuan

Tomlin

2017

2017 29th Chinese Control and Decision Conference (CCDC)

View full text Add to dashboard Cite

show abstract

“…where D p = ω p I (I is the identity matrix) and e p = y MF − Ry p . The idea is to force a quasi-Newton property on D p by setting either [13] v…”

Section: Lpic Detectors Based On Non-monotone Line-search Techniquesmentioning

confidence: 99%

“…The update equation for the LPIC detector based on BB iterative method is given by

{bold-italicy}_{p + 1} = {bold-italicy}_{p} + {bold-italicD}_{p} {bold-italice}_{p}

where D p = ω p I ( I is the identity matrix) and e p = y MF − Ry p . The idea is to force a quasi‐Newton property on D p by setting either [13]

ω_{p} = \underset{ω \in R}{\arg truemin} {∥D_{p}^{- 1} s_{p - 1} - v_{p - 1}∥}_{2}

ω_{p} = \underset{ω \in R}{\arg truemin} {∥s_{p - 1} - D_{p} v_{p - 1}∥}_{2}

where s p −1 = y p − y p −1 , v p −1 = g p − g p −1 and g p = − e p . The resulting weighting factors are the two well‐known BB weighting factors [9]

ω_{p}^{BB 1} = \frac{{bold-italics}_{p - 1}^{normalH} {bold-italics}_{p - 1}}{{bold-italics}_{p - 1}^{normalH} {bold-italicv}_{p - 1}} = \frac{g_{p - 1}^{H} g_{p - 1}}{g_{p - 1}^{H}}

…”

Section: Lpic Detectors Based On Non‐monotone Line‐search Techniquesmentioning

confidence: 99%

Competitive linear parallel interference cancellation detection based on monotone line‐search techniques

Bentrcia

Alshebeili

2014

IET signal process.

View full text Add to dashboard Cite

In this study, a competitive linear parallel interference cancellation (LPIC) detector that is memory efficient, enjoys fast convergence and low complexity and can be used to approximate the decorrelator/minimum-mean-square error detector in largescale communication systems is proposed. Similar to the non-monotone gradient-based LPIC detectors developed recently by Bentrcia and Alshebeili, the proposed detector maintains its efficiency and does not break down quickly if matrix-vector products are not performed accurately. However, unlike the previous detectors, the proposed detector relies on a monotone line-search technique which renders it more attractive because early stopping methods such as the L-curve method can be used to stop the LPIC iterations prior to convergence in order to avoid the noise enhancement effect. Simulation results agree well with the authors theoretical findings.

show abstract

New linear PIC detectors based on non-monotone line-search techniques

Bentrcia

Alshebeili

2012

Signal Processing

View full text Add to dashboard Cite

Quasi‐Newton Methods

Cited by 4 publications

References 62 publications

On the powerball method

On the powerball method

Competitive linear parallel interference cancellation detection based on monotone line‐search techniques

New linear PIC detectors based on non-monotone line-search techniques

Contact Info

Product

Resources

About