A new stopping criterion for Krylov solvers applied in Interior Point Methods

Abstract

A surprising result is presented in this paper with possible far reaching consequences for any optimization technique which relies on Krylov subspace methods employed to solve the underlying linear equation systems. In this paper the advantages of the new technique are illustrated in the context of Interior Point Methods (IPMs). When an iterative method is applied to solve the linear equation system in IPMs, the attention is usually placed on accelerating their convergence by designing appropriate preconditioners, but the linear solver is applied as a black box solver with a standard termination criterion which asks for a sufficient reduction of the residual in the linear system. Such an approach often leads to an unnecessary "oversolving" of linear equations. In this paper a new specialized termination criterion for Krylov methods used in IPMs is designed. It is derived from a deep understanding of IPM needs and is demonstrated to preserve the polynomial worst-case complexity of these methods. The new criterion has been adapted to the Conjugate Gradient (CG) and to the Minimum Residual method (MINRES) applied in the IPM context. The new criterion has been tested on a set of linear and quadratic optimization problems including compressed sensing, image processing and instances with partial differential equation constraints. Evidence gathered from these computational experiments shows that the new technique delivers significant improvements in terms of inner (linear) iterations and those translate into significant savings of the IPM solution time.

Proximal stabilized Interior Point Methods for quadratic programming and low-frequency-updates preconditioning techniques

“…Among other situations, this is the case of IPM matrices coming from problems of the form (20) for which H is diagonal or where H = 0, see, e.g., the LP case. In this section we prove a similar result to Lemma 3 when the involved matrices are the Schur complements of the linear systems ( 26) and (27). To this aim and for the sake of simplicity, we consider the case H = 0 and define L 1 (Θ) as the Schur complement of (26), i.e.,…”

“…In the following Remarks 1 and 2 we highlight the advantages given by the current reformulation of the Newton system showing, in essence, that the formulation in (26) allows better preconditioner re-usage than in the standard formulation (27). To this aim, as it is customary in IPM methods, let us suppose that…”

“…Analogously of what has been done in Section 6.1, we factorize a given L 1 (Θ) as (34) using Matlab's chol routine and, in our experiments, this factorization is recomputed if in the current PS-IPM step, PCG has performed more than 51% of the maximum allowed iterations in the solution of at least one of the two predictor-corrector systems. The stopping (absolute) tolerance for PCG is set as 10 −1 • tol (this choice does not guarantee in general the best performance, see [27] for a recent analysis, but it is a robust one).…”

Material-separating regularizer for multi-energy x-ray tomography

“…Next, we show in table 3 the results in terms of IPM iterations, PCG iterations and computational time, for various values of N. The IPM tolerance in ( 24) is set to 10 −8 ; we employed three centrality correctors with a symmetric neighbourhood (23) with parameter γ = 0.2. The linear system (21) is solved using PCG with a variable tolerance, chosen in order to reduce the number of inner iterations (see [34] for more details). Number of small elements and average product of g (1) and g (2) for different values of β; α = 500, N = 64.…”