Nearly Optimal Linear Convergence of Stochastic Primal-Dual Methods for Linear Programming

Abstract

There is a recent interest on first-order methods for linear programming (LP). In this paper, we propose a stochastic algorithm using variance reduction and restarts for solving sharp primaldual problems such as LP. We show that the proposed stochastic method exhibits a linear convergence rate for sharp instances with a high probability, which improves the complexity of the existing deterministic and stochastic algorithms. In addition, we propose an efficient coordinate-based stochastic oracle for unconstrained bilinear problems, which has O(1) per iteration cost and improves the total flop counts to reach a certain accuracy.

On the Infimal Sub-differential Size of Primal-Dual Hybrid Gradient Method

“…The first two examples, bilinear problem and linear programming, are direct consequences of the following lemmas [4,41]. Lemma 1.…”

“…Later, [10] presented a simplified and unified analysis for the ergodic rate of PDHG. More recently, many variants of PDHG have been proposed, including adaptive version [51,43,47,24] and stochastic version [8,1,41]. It was also shown that PDHG is equivalent to DRS up to a linear transformation [46,39].…”

“…Lemma 1. [41] Consider the convex-concave minimax problem min x∈X max y∈Y L(x, y). If for any z ∈ U(z * ) the minimax problem is sharp with constant η, i.e.…”