Unifying abstract inexact convergence theorems and block coordinate variable metric iPiano

Ochs, Peter

doi:10.48550/arxiv.1602.07283

Cited by 4 publications

(6 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the arising proximal mappings are computationally expensive. Extensions to a block-coordinate descent version are presented in [17] and a combination with an inertial method in [47]. In [6,7], the choice of the metric enjoys a great flexibility at the cost of an additional line search step in the algorithm.…”

Section: Related Workmentioning

confidence: 99%

Adaptive FISTA for Nonconvex Optimization

Ochs¹,

Pock²

2019

SIAM J. Optim.

Self Cite

View full text Add to dashboard Cite

In this paper we propose an adaptively extrapolated proximal gradient method, which is based on the accelerated proximal gradient method (also known as FISTA), however we locally optimize the extrapolation parameter by carrying out an exact (or inexact) line search. It turns out that in some situations, the proposed algorithm is equivalent to a class of SR1 (identity minus rank 1) proximal quasi-Newton methods. Convergence is proved in a general non-convex setting, and hence, as a byproduct, we also obtain new convergence guarantees for proximal quasi-Newton methods. The efficiency of the new method is shown in numerical experiments on a sparsity regularized non-linear inverse problem.

show abstract

Section: Related Workmentioning

confidence: 99%

Adaptive FISTA for Nonconvex Optimization

Ochs¹,

Pock²

2019

SIAM J. Optim.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Using this reformulation, they can apply classical Newton or quasi-Newton methods. Proximal quasi-Newton methods have also been considered in combination with the Heavy-ball method [49], and have been generalized further.…”

Section: Relation To Prior Workmentioning

confidence: 99%

On Quasi-Newton Forward-Backward Splitting: Proximal Calculus and Convergence

Becker¹,

Fadili²,

Ochs³

2019

SIAM J. Optim.

Self Cite

View full text Add to dashboard Cite

We introduce a framework for quasi-Newton forward-backward splitting algorithms (proximal quasi-Newton methods) with a metric induced by diagonal ± rank-r symmetric positive definite matrices. This special type of metric allows for a highly efficient evaluation of the proximal mapping. The key to this efficiency is a general proximal calculus in the new metric. By using duality, formulas are derived that relate the proximal mapping in a rank-r modified metric to the original metric. We also describe efficient implementations of the proximity calculation for a large class of functions; the implementations exploit the piece-wise linear nature of the dual problem. Then, we apply these results to acceleration of composite convex minimization problems, which leads to elegant quasi-Newton methods for which we prove convergence. The algorithm is tested on several numerical examples and compared to a comprehensive list of alternatives in the literature. Our quasi-Newton splitting algorithm with the prescribed metric compares favorably against state-of-the-art. The algorithm has extensive applications including signal processing, sparse recovery, machine learning and classification to name a few. Contributions.We introduce an general proximal calculus in a metric V = P ± Q ∈ S ++ (N ) given by P ∈ S ++ (N ) and a positive semi-definite rank-r matrix Q. This significantly extends the result in the preliminary version of this paper [7], where only V = P + Q with a rank-1 matrix Q is addressed. The general calculus is accompanied by several more concrete examples (see Section 3.3.4 for a non-exhaustive list), where, for example, the piecewise linear nature of certain dual problems is rigorously exploited.

show abstract

“…The following proof is now a slight adaptation of usual strategies for convergence under the K L-property [3] or [16,54], with the difference that we apply the K L-property to the set accum(u 0 ) instead of the set of critical points, which nevertheless fulfills E(u * ) < E(u k ) < E(u * )+η for any u * ∈ accum(u 0 ) as required in Lemma 6, due to the monotone descent of the algorithm. We then apply our previous results and find a global convergence in z k = ρ(u k ).…”

Section: Convergence Propertiesmentioning

confidence: 99%

“…For the special cases of induced squared norms, i.e. h = ||u|| 2 A , convergence still follows by adapting Section 3.3 to the results of the recent work [54], but we omit a further discussion.…”

Section: Inertiamentioning

confidence: 99%

Composite Optimization by Nonconvex Majorization-Minimization

Geiping¹,

Moeller²

2018

SIAM J. Imaging Sci.

View full text Add to dashboard Cite

The minimization of a nonconvex composite function can model a variety of imaging tasks. A popular class of algorithms for solving such problems are majorization-minimization techniques which iteratively approximate the composite nonconvex function by a majorizing function that is easy to minimize. Most techniques, e.g. gradient descent, utilize convex majorizers in order to guarantee that the majorizer is easy to minimize. In our work we consider a natural class of nonconvex majorizers for these functions, and show that these majorizers are still sufficient for a globally convergent optimization scheme. Numerical results illustrate that by applying this scheme, one can often obtain superior local optima compared to previous majorization-minimization methods, when the nonconvex majorizers are solved to global optimality. Finally, we illustrate the behavior of our algorithm for depth super-resolution from raw time-of-flight data.

show abstract

Unifying abstract inexact convergence theorems and block coordinate variable metric iPiano

Cited by 4 publications

References 0 publications

Adaptive FISTA for Nonconvex Optimization

Adaptive FISTA for Nonconvex Optimization

On Quasi-Newton Forward-Backward Splitting: Proximal Calculus and Convergence

Composite Optimization by Nonconvex Majorization-Minimization

Contact Info

Product

Resources

About