Adaptive mirror descent for constrained optimization

Bayandina, Anastasia

doi:10.1109/cnsa.2017.7973937

Cited by 3 publications

(6 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where the equality is a consequence of the definition of t * and the inequality is obtained (5) and 9, we obtain f…”

Section: Algorithm 1 Feasible Level-set Methodsmentioning

confidence: 92%

“…where the first inequality follows by (5); the second by (6); the third by (5); the fourth using (18) with k = K; the first equality by the geometric sum; and the fifth inequality by dropping negative terms.…”

Section: Overall Iteration Complexitymentioning

confidence: 99%

“…Ensuring feasibility using projection becomes difficult for general constraints (2). Some sub-gradient methods, including the method by Nesterov [29,Section 3.2.4] and its recent variants by Lan and Zhou [22] and Bayandina [5], can be applied to (1)- (2) without the need for projection mappings. Augmented Lagrangian methods have been recently developed by Xu [42,43,44] for solving (1)-(2) with additional linear equality constraints and can avoid projections by minimizing an augmented Lagrangian function that integrates both the constraint and objective functions.…”

mentioning

confidence: 99%

See 2 more Smart Citations

A Level-Set Method for Convex Optimization with a Feasible Solution Path

Lin¹,

Nadarajah²,

Soheili³

2018

SIAM J. Optim.

View full text Add to dashboard Cite

Large-scale constrained convex optimization problems arise in several application domains. First-order methods are good candidates to tackle such problems due to their low iteration complexity and memory requirement. The level-set framework extends the applicability of first-order methods to tackle problems with complicated convex objectives and constraint sets. Current methods based on this framework either rely on the solution of challenging subproblems or do not guarantee a feasible solution, especially if the procedure is terminated before convergence. We develop a level-set method that finds an-relative optimal and feasible solution to a constrained convex optimization problem with a fairly general objective function and set of constraints, maintains a feasible solution at each iteration, and only relies on calls to first-order oracles. We establish the iteration complexity of our approach, also accounting for the smoothness and strong convexity of the objective function and constraints when these properties hold. The dependence of our complexity on is similar to the analogous dependence in the unconstrained setting, which is not known to be true for level-set methods in the literature. Nevertheless, ensuring feasibility is not free. The iteration complexity of our method depends on a condition number, while existing level-set methods that do not guarantee feasibility can avoid such dependence. We numerically validate the usefulness of ensuring a feasible solution path by comparing our approach with an existing level set method on a Neyman-Pearson classification problem.

show abstract

“…where the equality is a consequence of the definition of t * and the inequality is obtained (5) and 9, we obtain f…”

Section: Algorithm 1 Feasible Level-set Methodsmentioning

confidence: 92%

Section: Overall Iteration Complexitymentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

A Level-Set Method for Convex Optimization with a Feasible Solution Path

Lin¹,

Nadarajah²,

Soheili³

2018

SIAM J. Optim.

View full text Add to dashboard Cite

show abstract

“…In comparison to known algorithms in the literature, the main advantage of our method for solving (1) is that the stopping criterion does not require the knowledge of constants M f , M g , and, in this sense, the method is adaptive. Mirror Descent with stepsizes not requiring knowledge of Lipschitz constants can be found, e.g., in [7] for problems without inequality constraints, and, for constrained problems, in [5].The algorithm is similar to the one in [2], but, for the sake of consistency with other parts of the chapter, we use slightly different proof.…”

Section: Convex Non-smooth Objective Functionmentioning

confidence: 99%

“…The idea of restarting a method for convex problems to obtain faster rate of convergence for strongly convex problems dates back to 1980's, see [19,20]. The algorithm is similar to the one in [2], but, for the sake of consistency with other parts of the chapter, we use slightly different proof. To show that restarting algorithm is also possible for problems with inequality constraints, we rely on the following lemma.…”

Section: Strongly Convex Non-smooth Objective Functionmentioning

confidence: 99%

Mirror Descent and Convex Optimization Problems with Non-smooth Inequality Constraints

Bayandina

Dvurechensky

Gasnikov

et al. 2018

Large-Scale and Distributed Optimization

Self Cite

View full text Add to dashboard Cite

We consider the problem of minimization of a convex function on a simple set with convex non-smooth inequality constraint and describe first-order methods to solve such problems in different situations: smooth or non-smooth objective function; convex or strongly convex objective and constraint; deterministic or randomized information about the objective and constraint. Described methods are based on Mirror Descent algorithm and switching subgradient scheme. One of our focus is to propose, for the listed different settings, a Mirror Descent with adaptive stepsizes and adaptive stopping rule. We also construct Mirror Descent for problems with objective function, which is not Lipschitz, e.g. is a quadratic function. Besides that, we address the question of recovering the dual solution in the considered problem.

show abstract