Quartic First-Order Methods for Low-Rank Minimization

Dragomir, Radu-Alexandru; d'Aspremont, Alexandre; Bolte, Jérôme

doi:10.48550/arxiv.1901.10791

Cited by 3 publications

(3 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Yet, even if f has those properties, g generally does not. For example, if f is quadratic then g(L, R) = f (LR ⊤ ) is quartic: this precludes Lipschitz continuity of the gradient (see Dragomir et al [17] for an interesting take on this issue). Moreover, the sublevel sets of g are never bounded because the fibers of the lift ϕ-that is, the sets of the form ϕ −1 (X) for X ∈ R m×n ≤r -are unbounded and g = f • ϕ is necessarily constant over those fibers.…”

Section: Optimization Through a Smooth Liftmentioning

confidence: 99%

Finding stationary points on bounded-rank matrices: a geometric hurdle and a smooth remedy

2022

View full text Add to dashboard Cite

We consider the problem of provably finding a stationary point of a smooth function to be minimized on the variety of bounded-rank matrices. This turns out to be unexpectedly delicate. We trace the difficulty back to a geometric obstacle: On a nonsmooth set, there may be sequences of points along which standard measures of stationarity tend to zero, but whose limit points are not stationary. We name such events apocalypses, as they can cause optimization algorithms to converge to non-stationary points. We illustrate this explicitly for an existing optimization algorithm on bounded-rank matrices. To provably find stationary points, we modify a trust-region method on a standard smooth parameterization of the variety. The method relies on the known fact that second-order stationary points on the parameter space map to stationary points on the variety. Our geometric observations and proposed algorithm generalize beyond boundedrank matrices. We give a geometric characterization of apocalypses on general constraint sets, which implies that Clarke-regular sets do not admit apocalypses. Such sets include smooth manifolds, manifolds with boundaries, and convex sets. Our trust-region method supports parameterization by any complete Riemannian manifold.

show abstract

Section: Optimization Through a Smooth Liftmentioning

confidence: 99%

Finding stationary points on bounded-rank matrices: a geometric hurdle and a smooth remedy

2022

View full text Add to dashboard Cite

show abstract

“…Practical examples of h-smoothness arise in Poisson inverse problems [3], quadratic inverse problems [8], rank minimization [12] and regularized higher-order tensor methods [31].…”

Section: Introductionmentioning

confidence: 99%

Optimal Complexity and Certification of Bregman First-Order Methods

Dragomir,

Taylor,

d'Aspremont

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

We provide a lower bound showing that the O(1/k) convergence rate of the NoLips method (a.k.a. Bregman Gradient or Mirror Descent) is optimal for the class of functions satisfying the h-smoothness assumption. This assumption, also known as relative smoothness, appeared in the recent developments around the Bregman Gradient method, where acceleration remained an open issue.On the way, we show how to constructively obtain the corresponding worst-case functions by extending the computer-assisted performance estimation framework of Drori and Teboulle (Mathematical Programming, 2014) to Bregman first-order methods, and to handle the classes of differentiable and strictly convex functions.

show abstract

“…Yet, even if f has those properties, g generally does not. For example, if f is quadratic then g(L, R) = f (LR ⊤ ) is quartic: this precludes Lipschitz continuity of the gradient (see Dragomir et al [18] for an interesting take on this issue). Moreover, the sublevel sets of g are never bounded because the fibers of the lift ϕ-that is, the sets of the form ϕ −1 (X) for X ∈ R m×n ≤r -are unbounded and g = f • ϕ is necessarily constant over those fibers.…”

Section: Characterization Of Apocalypsesmentioning

confidence: 99%

Finding stationary points on bounded-rank matrices: A geometric hurdle and a smooth remedy

Levin,

Kileel,

Boumal

2021

Preprint

View full text Add to dashboard Cite

We consider the problem of provably finding a stationary point of a smooth function to be minimized on the variety of bounded-rank matrices. While optimization on low-rank matrices has been extensively studied, existing algorithms do not provide such a basic guarantee. We trace this back to a geometric obstacle: On a nonsmooth set, there may be sequences of points along which standard measures of stationarity tend to zero, but whose limit point is not stationary. We name such events apocalypses, as they can cause optimization algorithms to converge to non-stationary points. We illustrate this on an existing algorithm using an explicit apocalypse on the bounded-rank matrix variety. To provably find stationary points, we modify a trust-region method on a standard smooth parameterization of the variety. The method relies on the known fact that second-order stationary points on the parameter space map to stationary points on the variety. Our geometric observations and proposed algorithm generalize beyond bounded-rank matrices. We give a geometric characterization of apocalypses on general constraint sets, which implies that Clarke-regular sets do not admit apocalypses. Such sets include smooth manifolds, manifolds with boundaries, and convex sets. Our trust-region method supports parameterization by any complete Riemannian manifold.

show abstract

Quartic First-Order Methods for Low-Rank Minimization

Cited by 3 publications

References 32 publications

Finding stationary points on bounded-rank matrices: a geometric hurdle and a smooth remedy

Finding stationary points on bounded-rank matrices: a geometric hurdle and a smooth remedy

Optimal Complexity and Certification of Bregman First-Order Methods

Finding stationary points on bounded-rank matrices: A geometric hurdle and a smooth remedy

Contact Info

Product

Resources

About