General-purpose preconditioning for regularized interior point methods

Gondzio, Jacek; Pougkakiotis, Spyridon; Pearson, John W.

doi:10.1007/s10589-022-00424-5

Cited by 3 publications

(2 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…as in [39,Section 3]). In particular, any preconditioner derived in [21] can be utilized for the proposed solver. However, for simplicity of exposition, we employ a standard factorization approach.…”

Section: The Ssn Linear Systemsmentioning

confidence: 99%

An active-set method for sparse approximations. Part II: General piecewise-linear terms

Pougkakiotis¹,

Gondzio²,

Kalogerias³

2023

Preprint

View full text Add to dashboard Cite

In this paper we present an efficient active-set method for the solution of convex quadratic programming problems with general piecewise-linear terms in the objective, with applications to sparse approximations and risk-minimization. The method exploits the structure of the piecewise-linear terms appearing in the objective in order to significantly reduce its memory requirements, and thus improve its efficiency. We showcase the robustness of the proposed solver on a variety of problems arising in risk-averse portfolio selection, quantile regression, and binary classification via linear support vector machines. We provide computational evidence to demonstrate, on real-world datasets, the ability of the solver of efficiently handling a variety of problems, by comparing it against an efficient general-purpose interior point solver as well as a state-of-the-art alternating direction method of multipliers. This work complements the accompanying paper ["An active-set method for sparse approximations. Part I: Separable ℓ1 terms", S. Pougkakiotis, J. Gondzio, D. S. Kalogerias], in which we discuss the case of separable ℓ1 terms, analyze the convergence, and propose general-purpose preconditioning strategies for the solution of its associated linear systems.Finally, it is important to note that model (P) allows for multiple piecewise-linear terms of the form max{Cx + d,since we can always adjust l to account for more than one terms. Hence, one can observe that (P) is quite general and can be used to model a plethora of very important problems that arise in practice.In light of the discussion in Remark 1, it is easily observed that problem (P) can model a plethora of very important problems arising in several application domains spanning, among others, operational research, machine learning, data science, and engineering. More specifically, various lasso and fussed lasso instances (with applications to sparse approximations for classification and regression [19,54], portfolio allocation [1], or medical diagnosis [23], among many others) can be readily modeled by (P). Additionally, various risk-minimization problems with linear random cost functions can be modeled by (P) (e.g. see [33,44,48]). Furthermore, even risk-minimization problems with nonlinear random cost functions, which are typically solved via Gauss-Newton schemes (e.g. see [10]), often require the solution of sub-problems like (P). Finally, continuous relaxations of integer programming problems with applications to operational research (e.g. [32]) often take the form of (P). Given the multitude of problems requiring easy access to (usually accurate) solutions of (P), the derivation of efficient, robust, and scalable solution methods is of paramount importance.Problem (P) can be solved by various first-or second-order methods. In particular, using a standard reformulation, by introducing several auxiliary variables, (P) can be written as a convex quadratic programming (QP) one and efficiently solved by, among others, an interior point method (IPM; e.g. [19,38]),...

show abstract

Section: The Ssn Linear Systemsmentioning

confidence: 99%

An active-set method for sparse approximations. Part II: General piecewise-linear terms

Pougkakiotis¹,

Gondzio²,

Kalogerias³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Active‐set methods are of high interest, especially when a sequence of related problems needs to be solved, because warm‐starts that reuse the solution from a previously solved problem can be easily accomplished. For approaches based on interior‐point methods (see, e.g., Reference 31), efficient warm‐starting is still an unsolved issue.…”

Section: Introductionmentioning

confidence: 99%

Double saddle‐point preconditioning for Krylov methods in the inexact sequential homotopy method

Pearson,

Potschka

2024

Numerical Linear Algebra App

View full text Add to dashboard Cite

We derive an extension of the sequential homotopy method that allows for the application of inexact solvers for the linear (double) saddle‐point systems arising in the local semismooth Newton method for the homotopy subproblems. For the class of problems that exhibit (after suitable partitioning of the variables) a zero in the off‐diagonal blocks of the Hessian of the Lagrangian, we propose and analyze an efficient, parallelizable, symmetric positive definite preconditioner based on a double Schur complement approach. For discretized optimal control problems with PDE constraints, this structure is often present with the canonical partitioning of the variables in states and controls. We conclude with numerical results for a badly conditioned and highly nonlinear benchmark optimization problem with elliptic partial differential equations and control bounds. The resulting method allows for the parallel solution of large 3D problems.

show abstract

An Interior Point-Proximal Method of Multipliers for Linear Positive Semi-Definite Programming

Pougkakiotis

Gondzio

2021

J Optim Theory Appl

View full text Add to dashboard Cite

In this paper we generalize the Interior Point-Proximal Method of Multipliers (IP-PMM) presented in Pougkakiotis and Gondzio (Comput Optim Appl 78:307–351, 2021. 10.1007/s10589-020-00240-9) for the solution of linear positive Semi-Definite Programming (SDP) problems, allowing inexactness in the solution of the associated Newton systems. In particular, we combine an infeasible Interior Point Method (IPM) with the Proximal Method of Multipliers (PMM) and interpret the algorithm (IP-PMM) as a primal-dual regularized IPM, suitable for solving SDP problems. We apply some iterations of an IPM to each sub-problem of the PMM until a satisfactory solution is found. We then update the PMM parameters, form a new IPM neighbourhood, and repeat this process. Given this framework, we prove polynomial complexity of the algorithm, under mild assumptions, and without requiring exact computations for the Newton directions. We furthermore provide a necessary condition for lack of strong duality, which can be used as a basis for constructing detection mechanisms for identifying pathological cases within IP-PMM.

show abstract

General-purpose preconditioning for regularized interior point methods

Cited by 3 publications

References 45 publications

An active-set method for sparse approximations. Part II: General piecewise-linear terms

An active-set method for sparse approximations. Part II: General piecewise-linear terms

Double saddle‐point preconditioning for Krylov methods in the inexact sequential homotopy method

An Interior Point-Proximal Method of Multipliers for Linear Positive Semi-Definite Programming

Contact Info

Product

Resources

About