Matrix completion with nonconvex regularization: spectral operators and scalable algorithms

Abstract

In this paper, we study the popularly dubbed matrix completion problem, where the task is to "fill in" the unobserved entries of a matrix from a small subset of observed entries, under the assumption that the underlying matrix is of low-rank. Our contributions herein, enhance our prior work on nuclear norm regularized problems for matrix completion (Mazumder et al., 2010) by incorporating a continuum of nonconvex penalty functions between the convex nuclear norm and nonconvex rank functions. Inspired by Soft-Impute (Mazumder et al., 2010;Hastie et al., 2016), we propose NC-Impute -an EM-flavored algorithmic framework for computing a family of nonconvex penalized matrix completion problems with warm-starts. We present a systematic study of the associated spectral thresholding operators, which play an important role in the overall algorithm. We study convergence properties of the algorithm. Using structured low-rank SVD computations, we demonstrate the computational scalability of our proposal for problems up to the Netflix size (approximately, a 500, 000 × 20, 000 matrix with 10 8 observed entries). We demonstrate that on a wide range of synthetic and real data instances, our proposed nonconvex regularization framework leads to low-rank solutions with better predictive performance when compared to those obtained from nuclear norm problems. Implementations of algorithms proposed herein, written in the R programming language, are made available on github.

Nonconvex matrix completion with Nesterov’s acceleration

“…where U and V is the left singular matrix and the right singular matrix of Z, σ is a singular value vector of Z and λ is a threshold. For the more general spectral penalty function P(σ ; λ, γ ), we obtain the following similar result [16] as Lemma 1.…”

“…It is critical to solve the SVD efficiently. Mazumder [16] et al used the alternating least square (ALS) procedure to compute a low-rank SVD.…”

“…Previous work [16] used the alternating least squares (ALS) procedure to optimize the nonconvex matrix completion. However, this method has high time complexities and requires more iterations to reach convergence, which cannot meet the requirements of big data analytics.…”

Matrix completion from a computational statistics perspective

“…Finally, the proximal mapping of a spectral function of a matrix X with SVD UΣV T is given by the following formula (Mazumder, Saldana, & Weng, , Proposition 1) …”

“…Finally, we point out that alternative sparsity inducing penalties can be employed. See, for example, the proximal maps of a variety of folded concave penalties discussed in Mazumder et al ().…”