Guarantees of Riemannian Optimization for Low Rank Matrix Recovery

Abstract

We establish theoretical recovery guarantees of a family of Riemannian optimization algorithms for low rank matrix recovery, which is about recovering an m × n rank r matrix from p < mn number of linear measurements. The algorithms are first interpreted as iterative hard thresholding algorithms with subspace projections. Based on this connection, we show that provided the restricted isometry constant R 3r of the sensing operator is less than C κ / √ r, the Riemannian gradient descent algorithm and a restarted variant of the Riemannian conjugate gradient algorithm are guaranteed to converge linearly to the underlying rank r matrix if they are initialized by one step hard thresholding. Empirical evaluation shows that the algorithms are able to recover a low rank matrix from nearly the minimum number of measurements necessary.

TARM: A Turbo-Type Algorithm for Affine Rank Minimization

“…(SVP) [7], normalized iterative hard thresholding [8], and Riemannian gradient descent (RGrad) [21]. We compare these algorithms under the same settings for 100 times, and the final results are averaged over all the comparisons.…”

“…These algorithms involve a projection step which projects a matrix into a low-rank space using truncated SVD. In [21], a Riemannian method, termed RGrad, was proposed to extend NIHT by projecting the search direction of gradient descent into a low dimensional space. Compared with the alternation minimization method, these IHT-based algorithms exhibit better convergence performance with lower computational complexity.…”

“…Compared with the alternation minimization method, these IHT-based algorithms exhibit better convergence performance with lower computational complexity. Furthermore, the convergence of these IHT-based algorithms are guaranteed when a certain restricted isometry property (RIP) holds [7], [8], [21]. In this paper, we aim to design low-complexity iterative algorithms to solve the stable ARM problem based on messagepassing principles [11], a perspective different from the existing approaches mentioned above.…”

Exploiting the Structure Effectively and Efficiently in Low-Rank Matrix Recovery

“…We will not discuss the approximate accuracy of Z 0 to X here; see [104,84,21] for related results. Starting from the spectral initialization, a sufficiently close initial guess can be constructed for PGD, and then exact recovery guarantee can be established.…”

“…where the new search P k is a weighted sum of the gradient descent direction G k and the previous search direction P k´1 . Several choices of the combination weight are available [104,100]. In each iteration, the Riemannian conjugate gradient descent algorithm has the same dominant computational cost as RGrad but with substantially faster convergence rate.…”