Stochastic First-Order Methods with Random Constraint Projection

Abstract

Abstract. We consider convex optimization problems with structures that are suitable for sequential treatment or online sampling. In particular, we focus on problems where the objective function is an expected value, and the constraint set is the intersection of a large number of simpler sets. We propose an algorithmic framework for stochastic first-order methods using random projection/proximal updates and random constraint updates, which contain as special cases several known algorithms as well as many new algorithms. To analyze the convergence of these algorithms in a unified manner, we prove a general coupled convergence theorem. It states that the convergence is obtained from an interplay between two coupled processes: progress toward feasibility and progress toward optimality. Under suitable stepsize assumptions, we show that the optimality error decreases at a rate of O(1/ √ k) and the feasibility error decreases at a rate of O(log k/k). We also consider a number of typical sampling processes for generating stochastic first-order information and random constraints, which are common in data-intensive applications, online learning, and simulation optimization. By using the coupled convergence theorem as a modular architecture, we are able to analyze the convergence of stochastic algorithms that use arbitrary combinations of these sampling processes.

Two stochastic optimization algorithms for convex optimization with fixed point constraints

“…Here, let us compare the stochastic first-order method with random constraint projection [45] with Algorithm 1. In [45], the problem…”

“…n ← n + 1 6: end loop Algorithms 1 and 2 are based on the Halpern fixed point algorithm [18,46]. In contrast to Algorithm 1, Algorithm 2 uses the approach of proximal point algorithms [2,Chapter 27], [3,5,31,32,40,45] that optimize nonsmooth, convex functions over the whole space.…”

“…Conditions (I)-(IV) were defined on the basis of Assumptions 4-7 in [45]. The conclusions in [45] show that the sequence (w n ) n∈N in each condition satisfies Sub-assumptions 3.2(i) and (ii). In the experiment, (w n ) n∈N in (IV) was generated using a positive Markov matrix with randomly chosen elements.…”

“…Stochastic approximation algorithms [13,14,29] perform convex stochastic composite optimization over a closed convex set. A distributed random projection algorithm [30] minimizes the sum of smooth, convex functions over the intersection of closed convex sets while incremental constraint projection-proximal methods [45] can be used to minimize the expected value of nonsmooth, convex functions over the intersection of closed convex sets. Multi-stage stochastic programming has been discussed [4,12,42,52], and the results [1,15,16,17,41,47] can even be applied to nonconvex stochastic optimization over the whole space or certain convex constraints.…”

“…This algorithm (Algorithm 1) blends a stochastic gradient method [6,Subchapter 10.2], [30,35,36,37,44] with the Halpern fixed point algorithm [18,46], which is a useful fixed point algorithm. Next, the nonsmooth convex stochastic programming problem is discussed (Section 4), and an algorithm (Algorithm 2) is presented that is based on the stochastic proximal point algorithm [3,5,45] and the Halpern fixed point algorithm.…”