Phase retrieval via randomized Kaczmarz: theoretical guarantees

Yan, Tan; Vershynin, Roman

doi:10.1093/imaiai/iay005

Cited by 85 publications

(85 citation statements)

References 25 publications

(48 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In particular, [127] demonstrates that the amplitude-based loss function has a better curvature, and vanilla gradient descent can indeed converge with a constant step size at the orderwise optimal sample complexity. A small sample of other nonconvex phase retrieval methods include [6,10,22,36,43,47,92,98,100,109,122], which are beyond the scope of this paper. • Matrix completion Nuclear norm minimization was studied in [19] as a convex relaxation paradigm to solve the matrix completion problem.…”

Section: Related Workmentioning

confidence: 99%

Implicit Regularization in Nonconvex Statistical Estimation: Gradient Descent Converges Linearly for Phase Retrieval, Matrix Completion, and Blind Deconvolution

et al. 2019

View full text Add to dashboard Cite

Recent years have seen a flurry of activities in designing provably efficient nonconvex procedures for solving statistical estimation problems. Due to the highly nonconvex nature of the empirical loss, state-of-the-art procedures often require proper regularization (e.g., trimming, regularized cost, projection) in order to guarantee fast convergence. For vanilla procedures such as gradient descent, however, prior theory either recommends highly conservative learning rates to avoid overshooting, or completely lacks performance guarantees. This paper uncovers a striking phenomenon in nonconvex optimization: even in the absence of explicit regularization, gradient descent enforces proper regularization implicitly under various statistical models. In fact, gradient descent follows a trajectory staying within a basin that enjoys nice geometry, consisting of points incoherent with the sampling mechanism. This "implicit regularization" feature allows gradient descent to proceed in a far more aggressive fashion without overshooting, which in turn results in substantial computational savings. Focusing on three fundamental statistical estimation problems, i.e., phase retrieval, low-rank matrix completion, and blind deconvolution, we establish that gradient descent achieves near-optimal statistical and computational guarantees without explicit regularization. In particular, by marrying statistical modeling with generic optimization theory, we develop a general recipe for analyzing the trajectories of iterative algorithms via a leave-one-out perturbation argument. As a by-product, for noisy matrix completion, we demonstrate that gradient descent achieves near-optimal error control-measured entrywise and by the spectral norm-which might be of independent interest.

show abstract

Section: Related Workmentioning

confidence: 99%

Implicit Regularization in Nonconvex Statistical Estimation: Gradient Descent Converges Linearly for Phase Retrieval, Matrix Completion, and Blind Deconvolution

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Moreover, under an appropriate model of the noise in the measurements, the problem is sharp [15,Propostion 3]. It is worthwhile to mention that numerous other approaches to phase retrieval exist, based on different problem formulations; see for example [4,5,34,36]. Experiment set-up: All of the experiments on phase retrieval will be generated according to the following procedure.…”

Section: Weakly Convex Functionsmentioning

confidence: 99%

Subgradient Methods for Sharp Weakly Convex Functions

Davis

Drusvyatskiy

MacPhee

et al. 2018

J Optim Theory Appl

View full text Add to dashboard Cite

Subgradient methods converge linearly on a convex function that grows sharply away from its solution set. In this work, we show that the same is true for sharp functions that are only weakly convex, provided that the subgradient methods are initialized within a fixed tube around the solution set. A variety of statistical and signal processing tasks come equipped with good initialization, and provably lead to formulations that are both weakly convex and sharp. Therefore, in such settings, subgradient methods can serve as inexpensive local search procedures. We illustrate the proposed techniques on phase retrieval and covariance estimation problems.

show abstract

“…In contrast, many simple nonconvex algorithms are able to solve (1) both accurately and efficiently. Among them are a family of algorithms with optimal or near-optimal provable guarantees, including alternating projections and its resampled variant [31,41], Kaczmarz methods [24,37], and those algorithms which propose to compute the solution of (1) by minimizing certain nonconvex loss functions [12,14,43,9,48]. Specifically, a gradient descent algorithm known as Wirtinger Flow has been developed in [12] based on the following smooth quadratic loss function…”

Section: Introductionmentioning

confidence: 99%

Toward the Optimal Construction of a Loss Function Without Spurious Local Minima for Solving Quadratic Equations

Cai

2020

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

The problem of finding a vector x which obeys a set of quadratic equations |a ⊤ k x| 2 = y k , k = 1, · · · , m, plays an important role in many applications. In this paper we consider the case when both x and a k are real-valued vectors of length n. A new loss function is constructed for this problem, which combines the smooth quadratic loss function with an activation function. Under the Gaussian measurement model, we establish that with high probability the target solution x is the unique local minimizer (up to a global phase factor) of the new loss function provided m n. Moreover, the loss function always has a negative directional curvature around its saddle points.

show abstract

Phase retrieval via randomized Kaczmarz: theoretical guarantees

Cited by 85 publications

References 25 publications

Implicit Regularization in Nonconvex Statistical Estimation: Gradient Descent Converges Linearly for Phase Retrieval, Matrix Completion, and Blind Deconvolution

Implicit Regularization in Nonconvex Statistical Estimation: Gradient Descent Converges Linearly for Phase Retrieval, Matrix Completion, and Blind Deconvolution

Subgradient Methods for Sharp Weakly Convex Functions

Toward the Optimal Construction of a Loss Function Without Spurious Local Minima for Solving Quadratic Equations

Contact Info

Product

Resources

About