Perturbed Proximal Descent to Escape Saddle Points for Non-convex and Non-smooth Objective Functions

Huang, Zhishen; Becker, Stephen

doi:10.1007/978-3-030-16841-4_7

Cited by 8 publications

(7 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…with regularization parameter λ > 0, which trades-off approximation error and sparsity. We can solve Equation ( 10) by a proximal gradient descent, as done in [66]; although problem ( 10) is non-convex, we have similar convergence results as for gradient descent, meaning that for small regularization parameters λ, we can show convergence of perturbed proximal gradient descent to a second-order stationary point [28].…”

Section: Setting Parameters To Zeromentioning

confidence: 79%

Best Approximate Quantum Compiling Problems

Madden

Simonetto

2022

ACM Transactions on Quantum Computing

View full text Add to dashboard Cite

We study the problem of finding the best approximate circuit that is the closest (in some pertinent metric) to a target circuit, and which satisfies a number of hardware constraints, like gate alphabet and connectivity. We look at the problem in the CNOT+rotation gate set from a mathematical programming standpoint, offering contributions both in terms of understanding the mathematics of the problem and its efficient solution. Among the results that we present, we are able to derive a 14-CNOT 4-qubit Toffoli decomposition from scratch, and show that the Quantum Shannon Decomposition can be compressed by a factor of two without practical loss of fidelity.

show abstract

Section: Setting Parameters To Zeromentioning

confidence: 79%

Best Approximate Quantum Compiling Problems

Madden

Simonetto

2022

ACM Transactions on Quantum Computing

View full text Add to dashboard Cite

show abstract

“…The convergence of these algorithms is usually established via the geometric analysis of the landscape of the objective function. One of the important geometric properties is the strict saddle property (Sun et al, 2018), which combined with the smoothness properties can guarantee the global polynomial-time convergence for various saddle-escaping algorithms (Jin et al, , 2018Sun et al, 2018;Huang & Becker, 2019). For the linear case, Ge et al (2016 proved the strict saddle property for both problems (3)-( 4) when the RIP constant is sufficiently small.…”

Section: Burer-monterio Factorization and Basic Propertiesmentioning

confidence: 99%

“…This result greatly improves the bounds in ; Zhu et al (2021) and extends the result in Ha et al (2020) to approximate second-order critical points. With the strict saddle property and certain smoothness properties, a wide range of algorithms guarantee a global polynomial-time convergence with a random initialization; see Jin et al ( , 2018; Sun et al (2018); Huang & Becker (2019). Due to the special non-convex structure of our problems and the RIP property, it is possible to prove the boundedness of the trajectory of the perturbed gradient descent method using a similar method as in .…”

Section: Contributionsmentioning

confidence: 99%

General Low-rank Matrix Optimization: Geometric Analysis and Sharper Bounds

Zhang¹,

Bi²,

Lavaei³

2021

Preprint

View full text Add to dashboard Cite

This paper considers the global geometry of general low-rank minimization problems via the Burer-Monterio factorization approach. For the rank-1 case, we prove that there is no spurious second-order critical point for both symmetric and asymmetric problems if the rank-2 RIP constant δ is less than 1/2. Combining with a counterexample with δ = 1/2, we show that the derived bound is the sharpest possible. For the arbitrary rank-r case, the same property is established when the rank-2r RIP constant δ is at most 1/3. We design a counterexample to show that the non-existence of spurious second-order critical points may not hold if δ is at least 1/2. In addition, for any problem with δ between 1/3 and 1/2, we prove that all second-order critical points have a positive correlation to the ground truth. Finally, the strict saddle property, which can lead to the polynomial-time global convergence of various algorithms, is established for both the symmetric and asymmetric problems when the rank-2r RIP constant δ is less than 1/3. The results of this paper significantly extend several existing bounds in the literature.Preprint. Under review.

show abstract

“…Iterative methods such as proximal gradient descent [33] and the alternating direction method of multipliers (ADMM) [11] are deployed for solving the aforementioned optimization problems. To minimize loss functions with nuclear norm, a surrogate relaxation and careful initialization is needed to deploy the conditional gradient method [100].…”

Section: Classical Model-based Image Reconstructionmentioning

confidence: 99%

Model-based Reconstruction with Learning: From Unsupervised to Supervised and Beyond

Huang,

Ye,

McCann

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Many techniques have been proposed for image reconstruction in medical imaging that aim to recover high-quality images especially from limited or corrupted measurements. Model-based reconstruction methods have been particularly popular (e.g., in magnetic resonance imaging and tomographic modalities) and exploit models of the imaging system's physics together with statistical models of measurements, noise and often relatively simple object priors or regularizers. For example, sparsity or low-rankness based regularizers have been widely used for image reconstruction from limited data such as in compressed sensing. Learning-based approaches for image reconstruction have garnered much attention in recent years and have shown promise across biomedical imaging applications. These methods include synthesis dictionary learning, sparsifying transform learning, and different forms of deep learning involving complex neural networks. We briefly discuss classical model-based reconstruction methods and then review reconstruction methods at the intersection of model-based and learning-based paradigms in detail. This review includes many recent methods based on unsupervised learning, and supervised learning, as well as a framework to combine multiple types of learned models together.

show abstract

Perturbed Proximal Descent to Escape Saddle Points for Non-convex and Non-smooth Objective Functions

Cited by 8 publications

References 13 publications

Best Approximate Quantum Compiling Problems

Best Approximate Quantum Compiling Problems

General Low-rank Matrix Optimization: Geometric Analysis and Sharper Bounds

Model-based Reconstruction with Learning: From Unsupervised to Supervised and Beyond

Contact Info

Product

Resources

About