A Level-Set Method for Convex Optimization with a Feasible Solution Path

Abstract

Large-scale constrained convex optimization problems arise in several application domains. First-order methods are good candidates to tackle such problems due to their low iteration complexity and memory requirement. The level-set framework extends the applicability of first-order methods to tackle problems with complicated convex objectives and constraint sets. Current methods based on this framework either rely on the solution of challenging subproblems or do not guarantee a feasible solution, especially if the procedure is terminated before convergence. We develop a level-set method that finds an-relative optimal and feasible solution to a constrained convex optimization problem with a fairly general objective function and set of constraints, maintains a feasible solution at each iteration, and only relies on calls to first-order oracles. We establish the iteration complexity of our approach, also accounting for the smoothness and strong convexity of the objective function and constraints when these properties hold. The dependence of our complexity on is similar to the analogous dependence in the unconstrained setting, which is not known to be true for level-set methods in the literature. Nevertheless, ensuring feasibility is not free. The iteration complexity of our method depends on a condition number, while existing level-set methods that do not guarantee feasibility can avoid such dependence. We numerically validate the usefulness of ensuring a feasible solution path by comparing our approach with an existing level set method on a Neyman-Pearson classification problem.

Contraction methods for continuous optimization

“…where A (k) f is the prolate series approximation (24). So it follows from the strong convergence Theorem 2.2, ( 19) and ( 25) that max…”

“…The concept of contractility comes from the level set. Actually, the problem (1) is closely related to the u-sublevel set [27,24,1], i.e., ( 2)…”

“…is the prolate series approximation given as (24) and N Ω,f = 2 s µ(Ω)ρ n /π n . Now, under the assumption of f * = 0, according to (10), the contracting sequence {D (k) } are defined recursively by D (0) = Ω and…”

First-Order Methods for Constrained Convex Programming Based on Linearized Augmented Lagrangian Function

“…The algorithms need O(ε − 1 2 ) iterations to achieve a primal ε solution, and the result matches with a lower complexity bound in Ouyang and Xu (2019) for bilinear SP problems. Another line of research is the level-set method of Aravkin et al (2019) and Lin et al (2018), which can also apply FOMs to each subproblem.…”

Stochastic First-order Methods for Convex and Nonconvex Functional Constrained Optimization

“…There exists a variety of literature on solving convex functional constrained optimization problems (1.1). One research line focuses on primal methods without involving the Lagrange multipliers including the cooperative subgradient methods [38,26] and level-set methods [27,34,29,4,28]. One possible limitation of these methods is the difficulty to directly achieve accelerated rate of convergence when the objective or constraint functions are smooth.…”