Sparse Matrix Methods in Optimization

Abstract

Abstract. Optimization algorithms typically require the solution of many systems of linear equations Bkyk b,. When large numbers of variables or constraints are present, these linear systems could account for much of the total computation time.Both direct and iterative equation solvers are needed in practice. Unfortunately, most of the off-the-shelf solvers are designed for single systems, whereas optimization problems give rise to hundreds or thousands of systems. To avoid refactorization, or to speed the convergence of an iterative method, it is essential to note that B is related to Bk_ 1.We review various sparse matrices that arise in optimization, and discuss compromises that are currently being made in dealing with them. Since significant advances continue to be made with single-system solvers, we give special attention to methods that allow such solvers to be used repeatedly on a sequence of modified systems (e.g., the product-form update; use of the Schur complement). The speed of factorizing a matrix then becomes relatively less important than the efficiency of subsequent solves with very many right-hand sides.At the same time, we hope that future improvements to linear-equation software will be oriented more specifically to the case of related matrices B k.

Optimization over the efficient set using an active constraint approach

“…This updating, which is similar to that used in the active constraint method solution of (NP2), can take advantage of the fact that one row is added and one row is removed from the constraint matrix and so requires only O(n 2) operations. On the other hand, refactorization of the matrices would require O(n 3) operations (e.g., see Gill and Murray [1978a, b] and Gill et al [1984]). …”

A direct method for sparse least squares problems with lower and upper bounds

“…In several sources of very large sparse linear least squares problems are identified and discussed. For a survey of methods for solving such problems see Bj6rck [3], Heath [18], and Gill et al [15].…”

An Algorithm for Large-Scale Quadratic Programming

“…The Schur-complement update method proposed by Gill et al (1985Gill et al ( , 1987a) manages to avoid this problem by maintaining a sparse factorization of the initial matrix X(/ (0) ) and a dense factorization of the Schur complement (see, for example, Golub & Van Loan, 1983) of /iC(/ (0) ) in a growing matrix of the form E r IE F Equations (2.6)-(2.8) and their solutions may be embedded in linear systems whose coefficient matrices are precisely of this form. The matrix E is made up from appropriate rows of the identity matrix and vectors aj.…”