We investigate and develop numerical methods for finite dimensional constrained structured optimization problems. Offering a comprehensive yet simple and expressive language, this problem class provides a modeling framework for a variety of applications. A general and flexible algorithm is proposed that interlaces proximal methods and safeguarded augmented Lagrangian schemes. We provide a theoretical characterization of the algorithm and its asymptotic properties, deriving convergence results for fully nonconvex problems. Adopting a proximal gradient method with an oracle as a formal tool, it is demonstrated how the inner subproblems can be solved by off-the-shelf methods for composite optimization, without introducing slack variables and despite the possibly set-valued projections. Finally, we describe our open-source matrix-free implementation ALPS of the proposed algorithm and test it numerically. Illustrative examples show the versatility of constrained structured programs as a modeling tool, expose difficulties arising in this vast problem class and highlight benefits of the implicit approach developed.

This paper is devoted to the theoretical and numerical investigation of an augmented Lagrangian method for the solution of optimization problems with geometric constraints. Specifically, we study situations where parts of the constraints are nonconvex and possibly complicated, but allow for a fast computation of projections onto this nonconvex set. Typical problem classes which satisfy this requirement are optimization problems with disjunctive constraints (like complementarity or cardinality constraints) as well as optimization problems over sets of matrices which have to satisfy additional rank constraints. The key idea behind our method is to keep these complicated constraints explicitly in the constraints and to penalize only the remaining constraints by an augmented Lagrangian function. The resulting subproblems are then solved with the aid of a problemtailored nonmonotone projected gradient method. The corresponding convergence theory allows for an inexact solution of these subproblems. Nevertheless, the overall algorithm computes so-called Mordukhovich-stationary points of the original problem under a mild asymptotic regularity condition, which is generally weaker than most of the respective available problem-tailored constraint qualifications. Extensive numerical experiments addressing complementarity-and cardinality-constrained optimization problems as well as a semidefinite reformulation of Maxcut problems visualize the power of our approach.

We consider a composite optimization problem where the sum of a continuously differentiable and a merely lower semicontinuous function has to be minimized. The proximal gradient algorithm is the classical method for solving such a problem numerically. The corresponding global convergence and local rate-of-convergence theory typically assumes, besides some technical conditions, that the smooth function has a globally Lipschitz continuous gradient and that the objective function satisfies the Kurdyka-Łojasiewicz property. Though this global Lipschitz assumption is satisfied in several applications where the objective function is, e.g., quadratic, this requirement is very restrictive in the nonquadratic case. Some recent contributions therefore try to overcome this global Lipschitz condition by replacing it with a local one, but, to the best of our knowledge, they still require some extra condition in order to obtain the desired global and rate-of-convergence results. The aim of this paper is to show that the local Lipschitz assumption together with the Kurdyka-Łojasiewicz property is sufficient to recover these convergence results.