Globalized inexact proximal Newton-type methods for nonconvex composite functions

Kanzow, Christian; Lechner, Theresa

doi:10.1007/s10589-020-00243-6

Cited by 25 publications

(39 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By combing this with the first inequality in (20), it is immediate to deduce that ϑ ′ k (t) > 0 for t ∈ (0, tk ) and ϑ ′ k (t) < 0 for t ∈ ( tk , t k − δ).…”

Section: Remark 31 (A)mentioning

confidence: 93%

“…To achieve the global convergence, a boundedness condition of the search directions (see (16)) is also required in [32,34,1]. We also notice that the inexact proximal Newton-type method in [22] was recently extended by Kanzow and Lechner [20] to solve the problem (2) with only a convex g, which essentially belongs to weakly convex optimization. Their global and local superlinear convergence results require the local strong convexity of Ψ around any stationary point.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

A globally convergent regularized Newton method for $\ell_q$-norm composite optimization problems

WU¹,

Pan²,

Yang³

2022

Preprint

View full text Add to dashboard Cite

This paper is concerned with ℓ q (0 < q < 1)-norm regularized minimization problems with a twice continuously differentiable loss function. For this class of nonconvex and nonsmooth composite problems, many algorithms have been proposed to solve them and most of which are of the first-order type. In this work, we propose a hybrid of proximal gradient method and subspace regularized Newton method, named HpgSRN. Unlike existing global convergent results for (regularized) Newton methods, the whole iterate sequence produced by HpgSRN is proved to have a finite length and converge to an L-type stationary point under a mild curve-ratio condition and the Kurdyka-Łojasiewicz property of the cost function, which does linearly if further a Kurdyka-Łojasiewicz property of exponent 1/2 holds. Moreover, a superlinear convergence rate for the iterate sequence is also achieved under an additional local error bound condition. Our convergence results do not require the isolatedness and strict local minimum properties of the L-stationary point. Numerical comparisons with ZeroFPR, a hybrid of proximal gradient method and quasi-Newton method for the forward-backward envelope of the cost function, proposed in [A.

show abstract

“…By combing this with the first inequality in (20), it is immediate to deduce that ϑ ′ k (t) > 0 for t ∈ (0, tk ) and ϑ ′ k (t) < 0 for t ∈ ( tk , t k − δ).…”

Section: Remark 31 (A)mentioning

confidence: 93%

Section: Related Workmentioning

confidence: 99%

A globally convergent regularized Newton method for $\ell_q$-norm composite optimization problems

WU¹,

Pan²,

Yang³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Indeed, (P) can model linear and convex quadratic programming instances, regularized (group) lasso instances (often arising in signal or image processing and machine learning, e.g. see [14,62,69]), as well as sub-problems arising from the linearization of a nonlinear (possibly non-convex or non-smooth) problem (such as those arising within sequential quadratic programming [10] or globalized proximal Newton methods [38,39]). Furthermore, various optimal control problems can be tackled in the form of (P), such as those arising from L 1 -regularized partial differential equation (PDE) optimization, assuming that a discretize-then-optimize strategy is adopted (e.g.…”

Section: Introductionmentioning

confidence: 99%

“…see [13,32,43,45,46,50,52,56,65]), variants of the proximal point method (e.g. see [20,24,25,38,39,41,49,59]), or interior point methods (IPMs) applied to a reformulation of (P) (e.g. see [21,28,31,51]).…”

Section: Introductionmentioning

confidence: 99%

“…Unlike most proximal point methods given in the literature (e.g. see the primal approaches in [38,39,49], the dual approaches in [40,41,73] or the primal-dual approaches in [30,20,25,59]), the proposed method is introducing proximal terms for each primal and dual variable of the problem, and this results in linear systems which are easy to precondition and solve, within the semi-smooth Newton method. Additionally, we explicitly deal with each of the two non-smooth terms of the objective in (P).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An interior point-proximal method of multipliers for convex quadratic programming

Pougkakiotis

Gondzio

2020

Comput Optim Appl

View full text Add to dashboard Cite

In this paper we combine an infeasible Interior Point Method (IPM) with the Proximal Method of Multipliers (PMM). The resulting algorithm (IP-PMM) is interpreted as a primal-dual regularized IPM, suitable for solving linearly constrained convex quadratic programming problems. We apply few iterations of the interior point method to each sub-problem of the proximal method of multipliers. Once a satisfactory solution of the PMM sub-problem is found, we update the PMM parameters, form a new IPM neighbourhood and repeat this process. Given this framework, we prove polynomial complexity of the algorithm, under standard assumptions. To our knowledge, this is the first polynomial complexity result for a primal-dual regularized IPM. The algorithm is guided by the use of a single penalty parameter; that of the logarithmic barrier. In other words, we show that IP-PMM inherits the polynomial complexity of IPMs, as well as the strict convexity of the PMM sub-problems. The updates of the penalty parameter are controlled by IPM, and hence are well-tuned, and do not depend on the problem solved. Furthermore, we study the behavior of the method when it is applied to an infeasible problem, and identify a necessary condition for infeasibility. The latter is used to construct an infeasibility detection mechanism. Subsequently, we provide a robust implementation of the presented algorithm and test it over a set of small to large scale linear and convex quadratic programming problems. The numerical results demonstrate the benefits of using regularization in IPMs as well as the reliability of the method.

show abstract

A Quasi-Newton Primal-Dual Algorithm with Line Search

Wang

Fadili

Ochs

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Quasi-Newton methods refer to a class of algorithms at the interface between first and second order methods. They aim to progress as substantially as second order methods per iteration, while maintaining the computational complexity of first order methods. The approximation of second order information by first order derivatives can be expressed as adopting a variable metric, which for (limited memory) quasi-Newton methods is of type "identity ± low rank". This paper continues the effort to make these powerful methods available for non-smooth systems occurring, for example, in large scale Machine Learning applications by exploiting this special structure. We develop a line search variant of a recently introduced quasi-Newton primal-dual algorithm, which adds significant flexibility, admits larger steps per iteration, and circumvents the complicated precalculation of a certain operator norm. We prove convergence, including convergence rates, for our proposed method and outperform related algorithms in a large scale image deblurring application. Keywords: quasi-Newton • primal-dual algorithm • line search • saddlepoint problems • large scale optimization ⋆ We acknowledge funding by the ANR-DFG joint project TRINOM-DS under the number DFG OC150/5-1.

show abstract

Globalized inexact proximal Newton-type methods for nonconvex composite functions

Cited by 25 publications

References 44 publications

A globally convergent regularized Newton method for $\ell_q$-norm composite optimization problems

A globally convergent regularized Newton method for $\ell_q$-norm composite optimization problems

An interior point-proximal method of multipliers for convex quadratic programming

A Quasi-Newton Primal-Dual Algorithm with Line Search

Contact Info

Product

Resources

About