Low-rank matrix completion using alternating minimization

Jain, Prateek; Netrapalli, Praneeth; Sanghavi, Sujay

doi:10.1145/2488608.2488693

Cited by 772 publications

(895 citation statements)

References 18 publications

Supporting

Mentioning

881

Contrasting

Order By: Relevance

“…Recently, there has been a resurgence of interest in methods based on alternating minimization, as numerous authors have shown that alternating minimization (suitably initialized, and under a few technical assumptions) provably converges to the global minimum for a range of problems including matrix completion [Kes12,JNS13,Har13], robust PCA [NNS + 14], and dictionary learning [AAJN13].…”

Section: Gordon's Generalizedmentioning

confidence: 99%

“…, n}, the problem (10) has no known analytical solution, but it is still easy to fit a model using alternating minimization. Algorithms based on alternating minimization have been shown to converge quickly (even geometrically [JNS13]) to a global solution satisfying a recovery guarantee when the initial values of X and Y are chosen carefully [KMO09, KMO10, KM10, JNS13, Har13, GAGG13].…”

Section: Missing Data and Matrix Completionmentioning

confidence: 99%

“…Algorithms based on alternating minimization have been shown to converge quickly (even geometrically [JNS13]) to a global solution satisfying a recovery guarantee when the initial values of X and Y are chosen carefully; see §2.4 for more details.…”

Section: Svdmentioning

confidence: 99%

“…What matrix corresponds to our data table? Here, we give a simple proposal for how to construct such a matrix, motivated by [KMO10,JNS13,Cha14]. Our key insight is that the SVD is the solution to our problem when the entries in the table have mean zero and variance one (and all the loss functions are quadratic).…”

Section: Svdmentioning

confidence: 99%

See 3 more Smart Citations

Generalized Low Rank Models

Udell

Horn

Zadeh

et al. 2016

FNT in Machine Learning

205

129

View full text Add to dashboard Cite

Principal components analysis (PCA) is a well-known technique for approximating a tabular data set by a low rank matrix. Here, we extend the idea of PCA to handle arbitrary data sets consisting of numerical, Boolean, categorical, ordinal, and other data types. This framework encompasses many well known techniques in data analysis, such as nonnegative matrix factorization, matrix completion, sparse and robust PCA, k-means, k-SVD, and maximum margin matrix factorization. The method handles heterogeneous data sets, and leads to coherent schemes for compressing, denoising, and imputing missing entries across all data types simultaneously. It also admits a number of interesting interpretations of the low rank factors, which allow clustering of examples or of features. We propose several parallel algorithms for fitting generalized low rank models, and describe implementations and numerical results.

show abstract

Section: Gordon's Generalizedmentioning

confidence: 99%

Section: Missing Data and Matrix Completionmentioning

confidence: 99%

Section: Svdmentioning

confidence: 99%

Section: Svdmentioning

confidence: 99%

See 2 more Smart Citations

Generalized Low Rank Models

Udell

Horn

Zadeh

et al. 2016

FNT in Machine Learning

205

129

View full text Add to dashboard Cite

show abstract

“…See, for example, the reports in [19,33,43,44,59,77,116,186,287]. For recent analysis of this method, see [136].…”

Section: Alternating Least Squares (Als)mentioning

confidence: 99%

Imputing Missing Entries of a Data Matrix: A Review

Dax¹

2014

JAC

View full text Add to dashboard Cite

The problem of imputing missing entries of a data matrix is easy to state: Some entries of the matrix are unknown and we want to assign "appropriate values" to these entries. The need for solving such problems arises in several applications, ranging from traditional ields to modern ones. Typical examples of traditional ields are statistical analysis of incomplete survey data, business reports, operations management, psychometrika, meteorology and hydrology. Modern applications arise in machine learning, data mining, DNA microarrays data, chemometrics, computer vision, recommender systems and collaborative iltering. The problem is highly interesting and challenging. Many ingenious algorithms have been proposed, and there is vast literature on imputing techniques. Yet, most of the papers consider the imputing problem within the context of a speci ic application or a speci ic approach. The current survey provides a broad view of the problem, one that exposes the large variety of imputing methods. Old and new methods are examined with focus on the ideas behind the methods. It is illustrated how simple ideas evolve into highly sophisticated algorithms. The review enables us to identify a number of basic concepts and tools which are shared by several imputing methods. Equivalence theorems that we prove reveal surprising relations between apparently different methods.

show abstract

Solving Random Quadratic Systems of Equations Is Nearly as Easy as Solving Linear Systems

Chen

Candès

2016

Comm Pure Appl Math

421

747

View full text Add to dashboard Cite

We consider the fundamental problem of solving quadratic systems of equations in n variables, where y i D jha i ; xij 2 , i D 1; : : : ; m, and x 2 R n is unknown. We propose a novel method, which starts with an initial guess computed by means of a spectral method and proceeds by minimizing a nonconvex functional as in the Wirtinger flow approach [13]. There are several key distinguishing features, most notably a distinct objective functional and novel update rules, which operate in an adaptive fashion and drop terms bearing too much influence on the search direction. These careful selection rules provide a tighter initial guess, better descent directions, and thus enhanced practical performance. On the theoretical side, we prove that for certain unstructured models of quadratic systems, our algorithms return the correct solution in linear time, i.e., in time proportional to reading the data fa i g and fy i g as soon as the ratio m=n between the number of equations and unknowns exceeds a fixed numerical constant. We extend the theory to deal with noisy systems in which we only have y i jha i ; xij 2 and prove that our algorithms achieve a statistical accuracy, which is nearly unimprovable. We complement our theoretical study with numerical examples showing that solving random quadratic systems is both computationally and statistically not much harder than solving linear systems of the same size-hence the title of this paper. For instance, we demonstrate empirically that the computational cost of our algorithm is about four times that of solving a least squares problem of the same size.

show abstract

Low-rank matrix completion using alternating minimization

Cited by 772 publications

References 18 publications

Generalized Low Rank Models

Generalized Low Rank Models

Imputing Missing Entries of a Data Matrix: A Review

Solving Random Quadratic Systems of Equations Is Nearly as Easy as Solving Linear Systems

Contact Info

Product

Resources

About