Matrix Rigidity and the Ill-Posedness of Robust PCA and Matrix Completion

Abstract

Robust Principal Component Analysis (PCA) (Candès et al., 2011) and low-rank matrix completion (Recht et al., 2010) are extensions of PCA that allow for outliers and missing entries respectively. It is well-known that solving these problems requires a low coherence between the low-rank matrix and the canonical basis, since in the extreme cases -when the low-rank matrix we wish to recover is also sparse -there is an inherent ambiguity. However, the well-posedness issue in both problems is an even more fundamental one: in some cases, both Robust PCA and matrix completion can fail to have any solutions due to the set of low-rank plus sparse matrices not being closed, which in turn is equivalent to the notion of the matrix rigidity function not being lower semicontinuous (Kumar et al., 2014). By constructing infinite families of matrices, we derive bounds on the rank and sparsity such that the set of low-rank plus sparse matrices is not closed. We also demonstrate numerically that a wide range of non-convex algorithms for both Robust PCA and matrix completion have diverging components when applied to our constructed matrices. An analogy can be drawn to the case of sets of higher order tensors not being closed under canonical polyadic (CP) tensor rank, rendering the best low-rank tensor approximation unsolvable (De Silva and Lim, 2008) and hence encourage the use of multilinear tensor rank (De Lathauwer, 2000).

Robust Matrix Completion By Exploiting Dynamic Low-Dimensional Structures

“…Fig. 1 shows the recovery performance of the convex relaxation (8) and Algorithm 1 with different d. We can see that Algorithm 1 performs better than the convex relaxation under the same d, and the performance of Algorithm 1 improves when d increases. When d = 5, the average running time of Algorithm 1 is 0.35 seconds, while the convex relaxation using CVX needs 300 seconds.…”

“…Fig. 2 shows the comparison of Algorithm 1 with the locally weighted matrix smoothing (LOWEMS) proposed in [21], and the convex relaxation (8). LOWEMS alternatively minimizes U and V in the objective function of equation ( 13) in [21] over U and V .…”

“…We first compare Algorithm 1 with the convex relaxation (8) and solve (8) by CVX [40]. We set n 1 = 50, n 2 = 50, r = 5 and constructL t ∈ R n1×n2 asL t = U (V t ) T whereŪ ∈ R n1×r andV t ∈ R n2×r .Ū andV 1 are matrices with i.i.d.…”

“…Many practical datasets have intrinsic low-dimensional structures despite the high ambient dimension. Motivated by applications like image and video processing [1,2], remote sensing [3] and collaborative filtering [4], the low-dimensional structures have been extensively studied in problems like Low-Rank Matrix Completion (LRMC) [5,6] and Robust Principal Component Analysis (RPCA) [7,8]. LRMC aims to recover the unobserved entries of a rank-r matrixL from the partial observations.…”

Compressed sensing of low-rank plus sparse matrices

“…Our contributions extend existing results on restricted isometry constants (RIC) for Gaussian and other measurement operators for sparse vectors [25] or low-rank matrices [11] to the sets of low-rank plus sparse matrices. For the set of matrices which are the sum of a low-rank plus a sparse matrix the results differ subtly due to the space not being closed, in that there are matrices X for which there does not exist a nearest projection to the set of low-rank plus sparse matrices [26]. To overcome this, we introduce the set of low-rank plus sparse matrices with a constraint on the Frobenius norm of its low-rank component, see Definition 1.1.…”