Asynchronous Stochastic Composition Optimization with Variance Reduction

Abstract

Composition optimization has drawn a lot of attention in a wide variety of machine learning domains from risk management to reinforcement learning. Existing methods solving the composition optimization problem often work in a sequential and single-machine manner, which limits their applications in large-scale problems. To address this issue, this paper proposes two asynchronous parallel variance reduced stochastic compositional gradient (AsyVRSC) algorithms that are suitable to handle large-scale data sets. The two algorithms are AsyVRSC-Shared for the shared-memory architecture and AsyVRSC-Distributed for the masterworker architecture. The embedded variance reduction techniques enable the algorithms to achieve linear convergence rates. Furthermore, AsyVRSC-Shared and AsyVRSC-Distributed enjoy provable linear speedup, when the time delays are bounded by the data dimensionality or the sparsity ratio of the partial gradients, respectively. Extensive experiments are conducted to verify the effectiveness of the proposed algorithms.

Numerical Methods for Distributed Stochastic Compositional Optimization Problems with Aggregative Structure

“…where the first equality follows from (25) and the row stochasticity of W, the first inequality follows from the fact |||W − I n ||| ≤ 2, the second inequality follows from Assumption 1(c), the fact…”

“…In the past decades, stochastic gradient decent methods have been well studied for solving stochastic compositional optimization problem, such as, two-timescale scheme method [28,29], sing-timescale scheme method [4,12] and variance reduction based method [14,17,25]. More recently, Gao and Huang [10] study distributed stochastic compositional optimization problem over undirected communication networks, where a gossip-based distributed stochastic gradient descent method and its gradient-tracking version are proposed.…”

Sample Complexity of Sample Average Approximation for Conditional Stochastic Optimization

“…When solving either (1.8) or (1.4), most of the existing work is devoted to developing stochastic oraclebased algorithms and their sample complexity analysis for solving these problems. Related work includes two-timescale stochastic approximation algorithms for solving the problem (1.8) (Wang et al, , 2016, variance-reduced algorithms for iteratively solving the SAA counterpart of (1.8) (Lian et al, 2017;Huo et al, 2018;Shen et al, 2018), and a primal-dual functional stochastic approximation algorithm for solving the problem (1.4) (Dai et al, 2017). These methods usually require convexity of the objective in order to obtain an -optimal solution.…”