The Global Optimization Geometry of Low-Rank Matrix Optimization

Zhu, Zhihui; Li, Qiuwei; Tang, Gongguo; Wakin, Michael B.

doi:10.1109/tit.2021.3049171

Cited by 34 publications

(25 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Third, we prove that in the exact parametrization case, problems (4)-( 5) both satisfy the strict saddle property (Sun et al, 2018) when the δ-RIP 2r,2r property is satisfied with δ < 1/3. This result greatly improves the bounds in ; Zhu et al (2021) and extends the result in Ha et al (2020) to approximate second-order critical points. With the strict saddle property and certain smoothness properties, a wide range of algorithms guarantee a global polynomial-time convergence with a random initialization; see Jin et al ( , 2018; Sun et al (2018); Huang & Becker (2019).…”

Section: Contributionssupporting

confidence: 69%

“…For the linear case, Ge et al (2016 proved the strict saddle property for both problems (3)-( 4) when the RIP constant is sufficiently small. More recently, Zhu et al (2021) extended the results to the nonlinear asymmetric case. Moreover, a weaker geometric property, namely the non-existence of spurious (non-global) second-order critical points, has been established for both problems when the RIP constant is small (Li et al, 2019;Ha et al, 2020).…”

Section: Burer-monterio Factorization and Basic Propertiesmentioning

confidence: 89%

“…Since the analysis of the statistical behavior of problems (3)-( 4) is not in the scope of this work, we focus on the case when the RIP property is satisfied by the objective function. We note that the RIP property is equivalent to the restricted strongly convex and smooth property defined in Wang et al (2017); Park et al (2018); Zhu et al (2021) with the condition number (1 + δ)/(1 − δ). Intuitively, the RIP property implies that the Hessian matrix is close to the identity tensor when the perturbation is restricted to be lowrank.…”

Section: Burer-monterio Factorization and Basic Propertiesmentioning

confidence: 90%

“…It has been proved in Zhu et al (2021) that the regularized problem (5) satisfies the strict saddle property if the function f a (•) has the δ-RIP 2r,4r property with…”

Section: Global Landscape: Strict Saddle Propertymentioning

confidence: 99%

See 3 more Smart Citations

General Low-rank Matrix Optimization: Geometric Analysis and Sharper Bounds

Zhang¹,

Bi²,

Lavaei³

2021

Preprint

View full text Add to dashboard Cite

This paper considers the global geometry of general low-rank minimization problems via the Burer-Monterio factorization approach. For the rank-1 case, we prove that there is no spurious second-order critical point for both symmetric and asymmetric problems if the rank-2 RIP constant δ is less than 1/2. Combining with a counterexample with δ = 1/2, we show that the derived bound is the sharpest possible. For the arbitrary rank-r case, the same property is established when the rank-2r RIP constant δ is at most 1/3. We design a counterexample to show that the non-existence of spurious second-order critical points may not hold if δ is at least 1/2. In addition, for any problem with δ between 1/3 and 1/2, we prove that all second-order critical points have a positive correlation to the ground truth. Finally, the strict saddle property, which can lead to the polynomial-time global convergence of various algorithms, is established for both the symmetric and asymmetric problems when the rank-2r RIP constant δ is less than 1/3. The results of this paper significantly extend several existing bounds in the literature.Preprint. Under review.

show abstract

Section: Contributionssupporting

confidence: 69%

Section: Burer-monterio Factorization and Basic Propertiesmentioning

confidence: 89%

Section: Burer-monterio Factorization and Basic Propertiesmentioning

confidence: 90%

“…It has been proved in Zhu et al (2021) that the regularized problem (5) satisfies the strict saddle property if the function f a (•) has the δ-RIP 2r,4r property with…”

Section: Global Landscape: Strict Saddle Propertymentioning

confidence: 99%

See 2 more Smart Citations

General Low-rank Matrix Optimization: Geometric Analysis and Sharper Bounds

Zhang¹,

Bi²,

Lavaei³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Theorem III.1 extends the results of Proposition 1 that hold at exact critical points to larger regions near critical points. This further establishes that the population risk (5) satisfies the robust strict saddle property and guarantees that many local search algorithms can in fact converge to local minima in polynomial time [26], [36], [37]. In summary, the population risk (5) has a favorable geometry.…”

Section: Resultsmentioning

confidence: 86%

Landscape Correspondence of Empirical and Population Risks in the Eigendecomposition Problem

Li,

Tang,

Wakin

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Spectral methods include a family of algorithms related to the eigenvectors of certain data-generated matrices. In this work, we are interested in studying the geometric landscape of the eigendecomposition problem in various spectral methods. In particular, we first extend known results regarding the landscape at critical points to larger regions near the critical points in a special case of finding the leading eigenvector of a symmetric matrix. For a more general eigendecomposition problem, inspired by recent findings on the connection between the landscapes of empirical risk and population risk, we then build a novel connection between the landscape of an eigendecomposition problem that uses random measurements and the one that uses the true data matrix. We also apply our theory to a variety of low-rank matrix optimization problems and conduct a series of simulations to illustrate our theoretical findings.

show abstract

Error bound of critical points and KL property of exponent 1/2 for squared F-norm regularized factorization

Tao

Pan

2021

J Glob Optim

View full text Add to dashboard Cite

The Global Optimization Geometry of Low-Rank Matrix Optimization

Cited by 34 publications

References 30 publications

General Low-rank Matrix Optimization: Geometric Analysis and Sharper Bounds

General Low-rank Matrix Optimization: Geometric Analysis and Sharper Bounds

Landscape Correspondence of Empirical and Population Risks in the Eigendecomposition Problem

Error bound of critical points and KL property of exponent 1/2 for squared F-norm regularized factorization

Contact Info

Product

Resources

About