Iterative Solution of Augmented Systems Arising in Interior Methods

Abstract

Abstract. Iterative methods are proposed for certain augmented systems of linear equations that arise in interior methods for general nonlinear optimization. Interior methods define a sequence of KKT equations that represent the symmetrized (but indefinite) equations associated with Newton's method for a point satisfying the perturbed optimality conditions. These equations involve both the primal and dual variables and become increasingly ill-conditioned as the optimization proceeds. In this context, an iterative linear solver must not only handle the ill-conditioning but also detect the occurrence of KKT matrices with the wrong matrix inertia. A one-parameter family of equivalent linear equations is formulated that includes the KKT system as a special case. The discussion focuses on a particular system from this family, known as the "doubly augmented system," that is positive definite with respect to both the primal and dual variables. This property means that a standard preconditioned conjugate-gradient method involving both primal and dual variables will either terminate successfully or detect if the KKT matrix has the wrong inertia. Constraint preconditioning is a well-known technique for preconditioning the conjugate-gradient method on augmented systems. A family of constraint preconditioners is proposed that provably eliminates the inherent ill-conditioning in the augmented system. A considerable benefit of combining constraint preconditioning with the doubly augmented system is that the preconditioner need not be applied exactly. Two particular "active-set" constraint preconditioners are formulated that involve only a subset of the rows of the augmented system and thereby may be applied with considerably less work. Finally, some numerical experiments illustrate the numerical performance of the proposed preconditioners and highlight some theoretical properties of the preconditioned matrices.

On mutual impact of numerical linear algebra and large-scale optimization with focus on interior point methods

“…A CP variant where D instead of H is approximated has been analysed in [28]; the resulting preconditioner is generally more expensive to apply, unless optimization problems with a particular structure are considered. The spectral properties of the preconditioned matrix P −1 K have been investigated in many papers [6,8,23,28,31,54,57]. The following result holds for the case D = 0 [54].…”

“…CPs date back to the 70's [1], but in the last decade they have been extensively applied to the KKT system, not only in the optimization context (see e.g. [7,8,13,23,24,28,31,45,47,54,57,70]). We consider the following symmetric indefinite CP:…”

Block triangular preconditioner for static Maxwell equations

“…The form of (1.2) frequently occurs in a large number of applications, such as the (linearized) Navier-Stokes equations [21], the time-harmonic Maxwell equations [7,8,10], the linear programming (LP) problem and the quadratic programming (QP) problem [17,20]. At present, there usually exist four kinds of preconditioners for the saddle point linear systems (1.2): block diagonal preconditioner [22,23,24,25], block triangular preconditioner [15,16,26,27,28,37], constraint preconditioner [29,30,31,32,33] and Hermitian and skew-Hermitian splitting (HSS) preconditioner [34]. One can [12] for a general discussion.…”

Using constraint preconditioners with regularized saddle-point problems

“…where G ∈ R n×n is some symmetric matrix, have been considered (for example, see [3,4,5,8,18,23].) When C = 0, (2) is commonly known as a constraint preconditioner [2,16,17,19].…”