A faster algorithm for solving general LPs

Jiang, Shunhua; Song, Zhao; Weinstein, Omri; Zhang, Hengjie

doi:10.1145/3406325.3451058

Cited by 29 publications

(10 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sketching and sampling are essential tools for speeding up computation costs of regression problems, we give a concise overview on the literature Sketching The celebrated sketch-and-solve paradigm is developed to speed up the computation of numerical linear algebra tasks, including regressions [2,15,34,35,4,36,1,48,47], low rank approximation [15,5,40,48], and more generally, sketching techniques have been useful in optimizations [30,25] and machines learning applications [1,9]. The most direct application of the sketch-and-solve paradigm is the (overcontrained) least squares regression problem, which aims to solve…”

Section: A Additional Related Workmentioning

confidence: 99%

Dynamic Least-Squares Regression

Jiang¹,

Peng²,

Weinstein³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

A common challenge in large-scale supervised learning, is how to exploit new incremental data to a pre-trained model, without re-training the model from scratch. Motivated by this problem, we revisit the canonical problem of dynamic least-squares regression (LSR), where the goal is to learn a linear model over incremental training data. In this setup, data and labels (A (t) , b (t) ) ∈ R t×d × R t evolve in an online fashion (t d), and the goal is to efficiently maintain an (approximate) solution to min x (t) A (t) x (t) − b (t) 2 for all t ∈ [T ]. Our main result is a dynamic data structure which maintains an arbitrarily small constant approximate solution to dynamic LSR with amortized update time O(d 1+o(1) ), almost matching the running time of the static (sketching-based) solution. By contrast, for exact (or even 1/ poly(n)-accuracy) solutions, we show a separation between the static and dynamic settings, namely, that dynamic LSR requires Ω(d 2−o(1) ) amortized update time under the OMv Conjecture (Henzinger et al., STOC'15). Our data structure is conceptually simple, easy to implement, and fast both in theory and practice, as corroborated by experiments over both synthetic and real-world datasets. 1

show abstract

Section: A Additional Related Workmentioning

confidence: 99%

Dynamic Least-Squares Regression

Jiang¹,

Peng²,

Weinstein³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Computation of the LM-cut heuristic takes a time at most quadratic in the size of the input planning problem (Helmert and Domshlak 2009). Our LP-TL model, on the other hand, uses a number of variables linear in size of the planning problem, which makes it impossible to be solved in quadratic time using the current algorithms (Jiang et al 2021). The situation for our LP-VE heuristic is even worse, as it may use a quadratic number of variables in the size of We have compared the time needed to compute h T L with the time needed for computing h LM -cut (Figure 3), and h V E (Figure 4), for all benchmark problems.…”

Section: Using H V E and H T L For Optimal Planningmentioning

confidence: 99%

Efficient Computation and Informative Estimation of h+ by Integer and Linear Programming

Rankooh¹,

Rintanen²

2022

ICAPS

View full text Add to dashboard Cite

We investigate modeling cost optimal delete-free STRIPS Planning by Integer/Linear Programming (IP/LP). We introduce two IP models and their LP relaxations based on a recently formulated representation of relaxed plans, named causal relaxed plan representation. The new models are produced by enforcing acyclicity in so-called causal relation graphs using vertex elimination and time labeling methods. We empirically show that while the vertex elimination based method outperforms the time labeling based method and all previously introduced domain independent methods for computing the exact values of h+, the time labeling based LP model is faster to solve compared to its vertex elimination based alternative, making it more suitable for using as heuristic function for optimal planning. We also theoretically analyze the admissible heuristic functions obtained by solving our LP models, and prove that the vertex elimination based heuristic is at least as informative as the time labeling based heuristic. Moreover, our empirical analysis shows that our vertex elimination based heuristic, which is a novel admissible estimation of h+, often has information complementary to that of the LM-cut heuristic.

show abstract

“…For ω ≈ 2.38 and α ≈ 0.31 this time complexity boils down to O n ω+o (1) . More recently, [39] reduced the running time of [19] to O (n ω + n 2.5−α/2+o (1) + n 2+1/18 ) , which further reduces the gap between matrix multiplication and solving LPs.…”

Section: Comparison With Related Workmentioning

confidence: 99%

Faster Randomized Interior Point Methods for Tall/Wide Linear Programs

Chowdhury¹,

Dexter²,

London³

et al. 2022

Preprint

View full text Add to dashboard Cite

Linear programming (LP) is an extremely useful tool which has been successfully applied to solve various problems in a wide range of areas, including operations research, engineering, economics, or even more abstract mathematical areas such as combinatorics. It is also used in many machine learning applications, such as 1 -regularized SVMs, basis pursuit, nonnegative matrix factorization, etc. Interior Point Methods (IPMs) are one of the most popular methods to solve LPs both in theory and in practice. Their underlying complexity is dominated by the cost of solving a system of linear equations at each iteration. In this paper, we consider both feasible and infeasible IPMs for the special case where the number of variables is much larger than the number of constraints. Using tools from Randomized Linear Algebra, we present a preconditioning technique that, when combined with the iterative solvers such as Conjugate Gradient or Chebyshev Iteration, provably guarantees that IPM algorithms (suitably modified to account for the error incurred by the approximate solver), converge to a feasible, approximately optimal solution, without increasing their iteration complexity. Our empirical evaluations verify our theoretical results on both real-world and synthetic data.

show abstract

A faster algorithm for solving general LPs

Cited by 29 publications

References 33 publications

Dynamic Least-Squares Regression

Dynamic Least-Squares Regression

Efficient Computation and Informative Estimation of h+ by Integer and Linear Programming

Faster Randomized Interior Point Methods for Tall/Wide Linear Programs

Contact Info

Product

Resources

About