From error bounds to the complexity of first-order descent methods for convex functions

Bolte, Jérôme; Nguyen, Trong Phong; Peypouquet, Juan; Suter, Bruce W.

doi:10.48550/arxiv.1510.08234

Cited by 8 publications

(23 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [16,17,18] the KL inequality was used to derive convergence rates of descent-type first order methods. The KL inequality was used to study convex optimization problems in [19,20].…”

Section: Introductionmentioning

confidence: 99%

Convergence rates of inertial splitting schemes for nonconvex composite optimization

Johnstone

Moulin

2017

2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

We study the convergence properties of a general inertial first-order proximal splitting algorithm for solving nonconvex nonsmooth optimization problems. Using the Kurdyka-Łojaziewicz (KL) inequality we establish new convergence rates which apply to several inertial algorithms in the literature. Our basic assumption is that the objective function is semialgebraic, which lends our results broad applicability in the fields of signal processing and machine learning. The convergence rates depend on the exponent of the "desingularizing function" arising in the KL inequality. Depending on this exponent, convergence may be finite, linear, or sublinear and of the form O(k −p ) for p > 1.

show abstract

“…In [16,17,18] the KL inequality was used to derive convergence rates of descent-type first order methods. The KL inequality was used to study convex optimization problems in [19,20].…”

Section: Introductionmentioning

confidence: 99%

Convergence rates of inertial splitting schemes for nonconvex composite optimization

Johnstone

Moulin

2017

2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…) for the denominator in (11). Now let 1 < α < 3 2 and put A = {(x, |x| α ) : x ∈ R}, so that A is above B and touches it at the origin.…”

Section: Slowly Shrinking Reachmentioning

confidence: 99%

“…Since A, B are convex, this is not surprising, as here linear convergence would require A, B to intersect at an angle, and not tangentially. For the Łojasiewicz exponent of i A + 1 2 d 2 B in the convex case see also [11], and for general considerations as to obtaining optimal θ see [17]. The corresponding global convergence theorem for the case r * = 0 is obtained in the same way using using the sets A s , B s and [30, Theorem 1], which leads to the rate b…”

Section: Convergencementioning

confidence: 99%

Alternating projections with applications to Gerchberg-Saxton error reduction

Noll¹

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Although it is hardly comprehensive, one example is noteworthy: property (2.1) holds if S = A * • ∇f • A for a strongly convex function f and a matrix A [25, p. 287]. See [7,38,24] for information on convex error bounds.…”

Section: Assumptions and Notationmentioning

confidence: 99%

“…The essential strong quasi-monotonicity property (2.1) appears to be the weakest possible condition under which a first-order algorithm will converge linearly. This deep, difficult to characterize property is related to the Hoffman bound [18], the linear regularity assumption, and the Kurdyka-Lojasiewicz property [7]. We look forward to a calculus of operations that preserve this property.…”

Section: ⊓ ⊔mentioning

confidence: 99%

SMART: The Stochastic Monotone Aggregated Root-Finding Algorithm

Davis

2016

Preprint

View full text Add to dashboard Cite

We introduce the Stochastic Monotone Aggregated Root-Finding (SMART) algorithm, a new randomized operator-splitting scheme for finding roots of finite sums of operators. These algorithms are similar to the growing class of incremental aggregated gradient algorithms, which minimize finite sums of functions; the difference is that we replace gradients of functions with black-boxes called operators, which represent subproblems to be solved during the algorithm. By replacing gradients with operators, we increase our modeling power, and we simplify the application and analysis of the resulting algorithms. The operator point of view also makes it easy to extend our algorithms to allow arbitrary sampling and updating of blocks of coordinates throughout the algorithm. Implementing and running an algorithm like this on a computing cluster can be slow if we force all computing nodes to be synched up at all times. To take better advantage of parallelism, we allow computing nodes to delay updates and break synchronization.This paper has several technical and practical contributions. We prove the weak, almost sure convergence of a new class of randomized operator-splitting schemes in separable Hilbert spaces; we prove that this class of algorithms convergences linearly in expectation when a weak regularity property holds; we highlight connections to other algorithms; and we introduce a few new algorithms for large-scale optimization.

show abstract

From error bounds to the complexity of first-order descent methods for convex functions

Cited by 8 publications

References 0 publications

Convergence rates of inertial splitting schemes for nonconvex composite optimization

Convergence rates of inertial splitting schemes for nonconvex composite optimization

Alternating projections with applications to Gerchberg-Saxton error reduction

SMART: The Stochastic Monotone Aggregated Root-Finding Algorithm

Contact Info

Product

Resources

About