Solving tall dense linear programs in nearly linear time

Abstract

In this paper we provide an O(nd + d 3 ) time randomized algorithm for solving linear programs with d variables and n constraints with high probability. To obtain this result we provide a robust, primal-dual O( √ d)-iteration interior point method inspired by the methods of Lee and Sidford (2014, 2019) and show how to efficiently implement this method using new data-structures based on heavy-hitters, the Johnson-Lindenstrauss lemma, and inverse maintenance. Interestingly, we obtain this running time without using fast matrix multiplication and consequently, barring a major advance in linear system solving, our running time is near optimal for solving dense linear programs among algorithms that don't use fast matrix multiplication.

Revisiting Tardos's Framework for Linear Programming: Faster Exact Solutions using Approximate Solvers

“…Fast algorithms withχ A dependence The layered least squares interior point methods discussed above represent substantial advances in the strongly polynomial solvability of LP, yet it is highly non-obvious how to combine these techniques with those of recent fast LP solvers. For example, for the results of [LS19,vdBLSS20], one would have to develop analogues of LLS steps for weighted versions of the logarithmic barrier. Furthermore, the proofs of exact convergence are intricate and deeply tied to the properties of the central path, and may leave one wondering whether theχ A solvability of LP is due to "IPM magic".…”

“…Harnessing the progress in approximate solvers The complexity of fast approximate LP algorithms has seen substantial improvements in recent years [LS19,CLS19,vdB20,vdBLSS20,LSZ19,JSWZ20]. Taking the recent algorithm [vdB20], given a feasible LP min c, x , Ax = b, x ≥ 0, having an optimal solution of 2 norm at most R, for ε > 0 it computes a pointx ≥ 0 satisfying c,x ≤ min in deterministic time O(n ω+o(1) log(n/ε)), where ω < 2.38 is the matrix multiplication exponent.…”

“…In Theorem 5.2 we use it for the special case c = 0; we now explain the more general case. We note that, instead of using the final output of Theorem 5.1, one could obtain more direct algorithms by using the interior-point methods of [LS19,vdB20,vdBLSS20] directly on our extended system. We now give a black-box argument using Theorem 5.1, to demonstrate the compatibility of our results with any approximate LP solver, not just interior-point methods.…”

Tutorial on the Robust Interior Point Method

“…Using further insights, their algorithm was derandomized by van den Brand [19]. The technique has been extended in subsequent papers to other optimization problems [14,7,23,8,20,6], and has also played a crucial role in faster algorithms for classical combinatorial optimization problems such as matchings and flows [22,21,4].…”

Minimum cost flows, MDPs, and ℓ 1 -regression in nearly linear time for dense instances

“…Recent advances in interior point methods (IPMs), a prominent class of continuous optimization methods, and data structures have led to nearly linear runtimes for solving fundamental classes of (1) to high precision with only a polylogarithmic dependence on problem parameters. In [BLSS20] an O(mn + n 2.5 ) time randomized method was obtained for solving (1) when ℓ i = 0 and u i = ∞ for all i ∈ [m], i.e., when the problem is in standard form. Further, in [BLN + 20] a randomized method was obtained for solving minimum cost perfect matching in bipartite graphs in time O(m + n 1.5 ), i.e.…”