Leave-One-Out Approach for Matrix Completion: Primal and Dual Analysis

Ding, Lijun; Chen, Yudong

doi:10.1109/tit.2020.2992769

Cited by 51 publications

(29 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The algorithm and theory that we develop are largely inspired by the recent advances of nonconvex optimization algorithms for low-rank matrix recovery problems (Keshavan et al 2010a,b;Candès et al 2015;Chen and Wainwright 2015;Sun and Luo 2016;Yi et al 2016;Chen and Candès 2017). The main theoretical tool-the leave-one-out analysis-is a powerful technique that has proved successful in various other statistical problems (El Karoui 2015, Abbe et al 2017, Ding and Chen 2018, Zhong and Boumal 2018, Chen et al 2019c, Chen et al 2019d, Chen et al 2019e, Li et al 2019, Pananjady and Wainwright 2019, Ma et al 2020). There are several major differences between the analysis of nonconvex tensor completion and that of nonconvex matrix recovery.…”

Section: Related Workmentioning

confidence: 99%

Nonconvex Low-Rank Tensor Completion from Noisy Data

Cai

Poor

et al. 2022

Operations Research

View full text Add to dashboard Cite

This paper investigates a problem of broad practical interest, namely, the reconstruction of a large-dimensional low-rank tensor from highly incomplete and randomly corrupted observations of its entries. Although a number of papers have been dedicated to this tensor completion problem, prior algorithms either are computationally too expensive for large-scale applications or come with suboptimal statistical performance. Motivated by this, we propose a fast two-stage nonconvex algorithm—a gradient method following a rough initialization—that achieves the best of both worlds: optimal statistical accuracy and computational efficiency. Specifically, the proposed algorithm provably completes the tensor and retrieves all low-rank factors within nearly linear time, while at the same time enjoying near-optimal statistical guarantees (i.e., minimal sample complexity and optimal estimation accuracy). The insights conveyed through our analysis of nonconvex optimization might have implications for a broader family of tensor reconstruction problems beyond tensor completion.

show abstract

Section: Related Workmentioning

confidence: 99%

Nonconvex Low-Rank Tensor Completion from Noisy Data

Cai

Poor

et al. 2022

Operations Research

View full text Add to dashboard Cite

show abstract

“…The existence of (approximate) dual certificates for matrix completion has been studied extensively [9,95,112,113]. We follow [121], which gives the current state-of-the-art sample complexity. The observation indices Ω are chosen randomly such that P((i, j) ∈ Ω) = p ∈ [0, 1) for all (i, j) independently.…”

Section: Definition 51 Given a Measurement Operatormentioning

confidence: 99%

“…[121] considers real matrices but the result can be easily extended to complex matrices 13. Meaning with probability at least 1 − c1(n1 + n2) −c 2 for constants c1, c2 > 0.…”

mentioning

confidence: 99%

WARPd: A linearly convergent first-order method for inverse problems with approximate sharpness conditions

Colbrook¹

2021

Preprint

View full text Add to dashboard Cite

Reconstruction of signals from undersampled and noisy measurements is a topic of considerable interest. Sharpness conditions directly control the recovery performance of restart schemes for firstorder methods without the need for restrictive assumptions such as strong convexity. However, they are challenging to apply in the presence of noise or approximate model classes (e.g., approximate sparsity). We provide a first-order method: Weighted, Accelerated and Restarted Primal-dual (WARPd), based on primal-dual iterations and a novel restart-reweight scheme. Under a generic approximate sharpness condition, WARPd achieves stable linear convergence to the desired vector. Many problems of interest fit into this framework. For example, we analyze sparse recovery in compressed sensing, low-rank matrix recovery, matrix completion, TV regularization, minimization of Bx l 1 under constraints (l 1 -analysis problems for general B), and mixed regularization problems. We show how several quantities controlling recovery performance also provide explicit approximate sharpness constants. Numerical experiments show that WARPd compares favorably with specialized state-of-the-art methods and is ideally suited for solving large-scale problems. We also present a noise-blind variant based on the Square-Root LASSO decoder. Finally, we show how to unroll WARPd as neural networks. This approximation theory result provides lower bounds for stable and accurate neural networks for inverse problems and sheds light on architecture choices. Code and a gallery of examples are made available online as a MATLAB package.

show abstract

“…Fortunately, there was a recent "breakthrough." Applying newer techniques such as the leave-oneout technique and fine-grained entry-wise analysis (Ma et al 2018, Ding and Chen 2020, Abbe et al 2020, Chen et al (2019Chen et al ( , 2020 proposed an uncertainty quantification technique for matrix completion, which satisfies the three "ideal" conditions above, in the case of homogeneous Gaussian noise. Further progress in Xia and Yuan (2021) extended this to homogeneous sub-Gaussian noise.…”

Section: Introductionmentioning

confidence: 99%

Uncertainty Quantification For Low-Rank Matrix Completion With Heterogeneous and Sub-Exponential Noise

Farias¹,

Li²,

Peng³

2021

Preprint

View full text Add to dashboard Cite

The problem of low-rank matrix completion with heterogeneous and sub-exponential (as opposed to homogeneous and Gaussian) noise is particularly relevant to a number of applications in modern commerce. Examples include panel sales data and data collected from web-commerce systems such as recommendation engines. An important unresolved question for this problem is characterizing the distribution of estimated matrix entries under common low-rank estimators. Such a characterization is essential to any application that requires quantification of uncertainty in these estimates and has heretofore only been available under the assumption of homogenous Gaussian noise. Here we characterize the distribution of estimated matrix entries when the observation noise is heterogeneous sub-exponential and provide, as an application, explicit formulas for this distribution when observed entries are Poisson or Binary distributed.

show abstract

Leave-One-Out Approach for Matrix Completion: Primal and Dual Analysis

Cited by 51 publications

References 18 publications

Nonconvex Low-Rank Tensor Completion from Noisy Data

Nonconvex Low-Rank Tensor Completion from Noisy Data

WARPd: A linearly convergent first-order method for inverse problems with approximate sharpness conditions

Uncertainty Quantification For Low-Rank Matrix Completion With Heterogeneous and Sub-Exponential Noise

Contact Info

Product

Resources

About