Inexact penalty decomposition methods for optimization problems with geometric constraints

Abstract

This paper provides a theoretical and numerical investigation of a penalty decomposition scheme for the solution of optimization problems with geometric constraints. In particular, we consider some situations where parts of the constraints are nonconvex and complicated, like cardinality constraints, disjunctive programs, or matrix problems involving rank constraints. By a variable duplication and decomposition strategy, the method presented here explicitly handles these difficult constraints, thus generating iterates which are feasible with respect to them, while the remaining (standard and supposingly simple) constraints are tackled by sequential penalization. Inexact optimization steps are proven sufficient for the resulting algorithm to work, so that it is employable even with difficult objective functions. The current work is therefore a significant generalization of existing papers on penalty decomposition methods. On the other hand, it is related to some recent publications which use an augmented Lagrangian idea to solve optimization problems with geometric constraints. Compared to these methods, the decomposition idea is shown to be numerically superior since it allows much more freedom in the choice of the subproblem solver, and since the number of certain (possibly expensive) projection steps is significantly less. Extensive numerical results on several highly complicated classes of optimization problems in vector and matrix spaces indicate that the current method is indeed very efficient to solve these problems.

On the Convergence of Inexact Alternate Minimization in Problems with ℓ0 Penalties

“…In conclusion, similarly as what was shown for the case of cardinality constrained problems [16,17], we have basically proved here that the PD scheme can be employed, with no loss of convergence guarantees, solving the subproblems by inexact AM; this result also justifies the use of the PD method in the very common settings where the function f is not convex.…”

“…We can note that the unconstrained minimization of function ( 3) perfectly fits the framework (1) considered in this work: function [16,17] and y-update subproblem can be solved in closed form up to global optimality [19].…”