Generalized Low Rank Models

Udell, Madeleine; Horn, Corinne; Zadeh, Reza; Boyd, Stephen

doi:10.1561/9781680831412

Cited by 87 publications

(86 citation statements)

References 115 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The global geometry of low-rank matrix recovery but with analysis customized to linear measurements and quadratic loss functions is also covered in [34], [41]. Since Theorem 1 only requires the (2r, 4r)-restricted strong convexity and smoothness property (6), aside from low-rank matrix recovery [45], it can also be applied to many other low-rank matrix optimization problems [46] which do not necessarily involve quadratic loss functions. Typical examples include 1-bit matrix completion [43], [47] and Poisson principal component analysis (PCA) [48].…”

Section: B Global Geometry For General Low-rank Optimizationmentioning

confidence: 99%

Global optimality in low-rank matrix optimization

Zhu¹,

Li²,

Tang³

et al. 2017

2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP)

View full text Add to dashboard Cite

This paper considers general rank-constrained optimization problems that minimize a general objective function f (X) over the set of rectangular n × m matrices that have rank at most r. To tackle the rank constraint and also to reduce the computational burden, we factorize X into U V T where U and V are n × r and m × r matrices, respectively, and then optimize over the small matrices U and V . We characterize the global optimization geometry of the nonconvex factored problem and show that the corresponding objective function satisfies the robust strict saddle property as long as the original objective function f satisfies restricted strong convexity and smoothness properties, ensuring global convergence of many local search algorithms (such as noisy gradient descent) in polynomial time for solving the factored problem. We also provide a comprehensive analysis for the optimization geometry of a matrix factorization problem where we aim to find n × r and m × r matrices U and V such that U V T approximates a given matrix X . Aside from the robust strict saddle property, we show that the objective function of the matrix factorization problem has no spurious local minima and obeys the strict saddle property not only for the exact-parameterization case where rank(X ) = r, but also for the over-parameterization case where rank(X ) < r and the under-parameterization case where rank(X ) > r. These geometric properties imply that a number of iterative optimization algorithms (such as gradient descent) converge to a global solution with random initialization.

show abstract

Section: B Global Geometry For General Low-rank Optimizationmentioning

confidence: 99%

Global optimality in low-rank matrix optimization

Zhu¹,

Li²,

Tang³

et al. 2017

2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP)

View full text Add to dashboard Cite

show abstract

“…In addition, our algorithmic framework also allows for direct incorporation of constraints not only on estimates of the matrix factors, but also on the estimate of X * itself to account for entry-wise structural constraints that could arise naturally in many matrix completion scenarios. Several other recent efforts in the DL literature have proposed algorithmic procedures for coping with missing data [37], [38], and a survey of algorithmic approaches to generalized low-rank modeling tasks is given in the recent work [39].…”

Section: B Connections With Existing Workmentioning

confidence: 99%

Noisy Matrix Completion Under Sparse Factor Models

Soni

Jain

Haupt

et al. 2016

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

This paper examines a general class of noisy matrix completion tasks where the goal is to estimate a matrix from observations obtained at a subset of its entries, each of which is subject to random noise or corruption. Our specific focus is on settings where the matrix to be estimated is well-approximated by a product of two (a priori unknown) matrices, one of which is sparse. Such structural models -referred to here as "sparse factor models" -have been widely used, for example, in subspace clustering applications, as well as in contemporary sparse modeling and dictionary learning tasks. Our main theoretical contributions are estimation error bounds for sparsity-regularized maximum likelihood estimators for problems of this form, which are applicable to a number of different observation noise or corruption models. Several specific implications are examined, including scenarios where observations are corrupted by additive Gaussian noise or additive heavier-tailed (Laplace) noise, Poisson-distributed observations, and highly-quantized (e.g., one-bit) observations. We also propose a simple algorithmic approach based on the alternating direction method of multipliers for these tasks, and provide experimental evidence to support our error analyses.

show abstract

“…But we will not follow the general algorithm proposed in [21] for the following reasons: (i) due to the special structure of the weight, our algorithm is more efficient than [21] (see Algorithm 3.1, in p. 42 [21]), (ii) it allows a detailed convergence analysis which is usually not available in other algorithms proposed in the literature [9,26,21], and (iii) it can handle bigger size matrices as we will demonstrate in the numerical result section. If k = 0, then (4.1) reduces to an unweighted rank r factorization of A 2 and can be solved as an alternating least squares problem [16,17,18].…”

Section: )mentioning

confidence: 99%

On a Problem of Weighted Low-Rank Approximation of Matrices

Dutta¹,

Li²

2017

SIAM J. Matrix Anal. & Appl.

View full text Add to dashboard Cite

We study a weighted low rank approximation that is inspired by a problem of constrained low rank approximation of matrices as initiated by the work of Golub, Hoffman, and Stewart (Linear Algebra and Its Applications, 88-89(1987), 317-327). Our results reduce to that of Golub, Hoffman, and Stewart in the limiting cases. We also propose an algorithm based on the alternating direction method to solve our weighted low rank approximation problem and compare it with the state-of-art general algorithms such as the weighted total alternating least squares and the EM algorithm.

show abstract

Generalized Low Rank Models

Cited by 87 publications

References 115 publications

Global optimality in low-rank matrix optimization

Global optimality in low-rank matrix optimization

Noisy Matrix Completion Under Sparse Factor Models

On a Problem of Weighted Low-Rank Approximation of Matrices

Contact Info

Product

Resources

About