General Low-rank Matrix Optimization: Geometric Analysis and Sharper Bounds

Abstract

This paper considers the global geometry of general low-rank minimization problems via the Burer-Monterio factorization approach. For the rank-1 case, we prove that there is no spurious second-order critical point for both symmetric and asymmetric problems if the rank-2 RIP constant δ is less than 1/2. Combining with a counterexample with δ = 1/2, we show that the derived bound is the sharpest possible. For the arbitrary rank-r case, the same property is established when the rank-2r RIP constant δ is at most 1/3. We design a counterexample to show that the non-existence of spurious second-order critical points may not hold if δ is at least 1/2. In addition, for any problem with δ between 1/3 and 1/2, we prove that all second-order critical points have a positive correlation to the ground truth. Finally, the strict saddle property, which can lead to the polynomial-time global convergence of various algorithms, is established for both the symmetric and asymmetric problems when the rank-2r RIP constant δ is less than 1/3. The results of this paper significantly extend several existing bounds in the literature.Preprint. Under review.

Local and Global Linear Convergence of General Low-rank Matrix Recovery Problems

“…it is clear that ∇ 2 F is ρ 2 /2-restricted Lipschitz continuous as the second term in the above equation is independent of N . Next, we can repeat the argument in Lemma 9 with the function f s replaced with 4F/(1 + δ), noting that the latter function satisfies the 2δ/(1 + δ)-RIP 2r property as proven in Theorem 12 of Zhang et al (2021).…”

“…For the asymmetric problem, it can be verified that the function F in (7) satisfies the 2δ/(1 + δ)-RIP 2r property after scaling (see Zhang et al (2021)). Furthermore, if M * = U * V * T is a balanced factorization of the ground truth M * , then…”

“…For general symmetric and asymmetric problems with arbitrary objectives and rank r, we will utilize the following result that serves a similar purpose as Lemma 4, which is a consequence of Theorem 6 and Theorem 7 in Zhang et al (2021). Lemma 6.…”

“…Since the objective functions g s (X) and g a (X) are nonconvex, local search methods may converge to a spurious (nonglobal) local minimum. To guarantee the absence of such spurious solutions, the restricted isometry property (RIP) defined below is the most common condition imposed on the functions f s and f a used in the literature such as Bhojanapalli et al (2016b); Ge et al (2017); Zhu et al (2018); Zhang et al (2018bZhang et al ( ,a, 2019; Ha et al (2020); Zhang and Zhang (2020); Bi and Lavaei (2020); Zhang et al (2021); Zhang (2021).…”

“…The state-of-the-art results on the nonexistence of spurious local minima are presented in Zhang et al (2021); Zhang (2021). Zhang et al (2021) shows that the problem (4) or ( 5) is devoid of spurious local minima if i) the associated function f s or f a satisfies the δ-RIP 2 property with δ < 1/2 in case r = 1, ii) the function f s or f a satisfies the δ-RIP 2r property with δ ≤ 1/3 in case r > 1.…”