Linear solvers for power grid optimization problems: A review of GPU-accelerated linear solvers

Świrydowicz, Kasia; Darve, Eric; Jones, Wesley B.; Maack, Jonathan; Regev, Shaked; Saunders, Michael A.; Thomas, Stephen J.; Peleš, Slaven

doi:10.1016/j.parco.2021.102870

Cited by 30 publications

(13 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…LDL T factorization via MA57 [14] has been used effectively for extremely sparse problems on traditional CPU-based platforms, but is not suitable for fine grain parallelization required for GPU acceleration. Parallel and GPU accelerated direct solve implementations such as SuperLU [1,25], STRUMPACK [39,45], and PaStiX [20,32] exist for general symmetric indefinite systems (although the first two are designed for general systems), but these software packages are designed to take advantage of dense blocks of the matrices in denser problems and do not perform well on our systems of interest, which do not yield these dense blocks [19,46].…”

Section: Solving Kkt Linear Systemsmentioning

confidence: 99%

“…literature showing that multifrontal or supernodal approaches are not suitable for very sparse and irregular systems, where the dense blocks become too small, leading to an unfavorable ratio of communication versus computation [7,12,19]. This issue is exacerbated when supernodal or multifrontal approaches are used for fine-grain parallelization on GPUs [46]. Our method becomes better when the ratio of LDL T to Cholesky factorization time grows, because factorization is the most costly part of linear solvers and our method has more (but smaller and less costly) system solves.…”

Section: Comparison With Ldl Tmentioning

confidence: 99%

“…(Otherwise, interior methods perform curvature tests to ensure descent in a certain merit function [11].) Relatively few linear solvers CONTACT Shaked Regev Email: sregev@stanford.edu are equipped to solve KKT systems, and even fewer to run those computations on hardware accelerators such as graphic processing units (GPUs) [46].…”

Section: Introductionmentioning

confidence: 99%

“…Unfortunately, LDL T factorization is unstable without pivoting. This is why LDL T approaches, typically used by interior methods for nonlinear problems on CPU-based platforms [47], have not performed as well as on hardware accelerators [46]. Some iterative methods such as MINRES [31] for general symmetric matrices can make efficient (albeit memory bandwidth limited) use of GPUs because they only require matrixvector multiplications at each iteration, which can be highly optimized, but they have limited efficiency when the number of iterations becomes large.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A Hybrid Direct-Iterative Method for Solving KKT Linear Systems

Regev,

Chiang,

Darve

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We propose a solution strategy for linear systems arising in interior method optimization, which is suitable for implementation on hardware accelerators such as graphical processing units (GPUs). The current gold standard for solving these systems is the LDL T factorization. However, LDL T requires pivoting during factorization, which substantially increases communication cost and degrades performance on GPUs. Our novel approach solves a large indefinite system by solving multiple smaller positive definite systems, using an iterative solve for the Schur complement and an inner direct solve (via Cholesky factorization) within each iteration. Cholesky is stable without pivoting, thereby reducing communication and allowing reuse of the symbolic factorization. We demonstrate the practicality of our approach and show that on large systems it can efficiently utilize GPUs and outperform LDL T factorization of the full system.

show abstract

Section: Solving Kkt Linear Systemsmentioning

confidence: 99%

Section: Comparison With Ldl Tmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Hybrid Direct-Iterative Method for Solving KKT Linear Systems

Regev,

Chiang,

Darve

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Unfortunately, factorization of unstructured sparse indefinite matrix is one of these edge cases: unlike dense matrices, sparse matrices have unstructured sparsity, rendering most sparse algorithms difficult to parallelize. Thus, implementing a sparse direct solver on the GPU is nontrivial, and the perfor-mance of current GPU-based sparse linear solvers lags far behind that of their CPU equivalents [36,37]. Previous attempts to solve nonlinear problems on the GPU have circumvented this problem by relying on iterative solvers [7,33] or on decomposition methods [23].…”

Section: Introductionmentioning

confidence: 99%

Condensed interior-point methods: porting reduced-space approaches on GPU hardware

Pacaud¹,

Shin²,

Schanen³

et al. 2022

Preprint

View full text Add to dashboard Cite

The interior-point method (IPM) has become the workhorse method for nonlinear programming. The performance of IPM is directly related to the linear solver employed to factorize the Karush-Kuhn-Tucker (KKT) system at each iteration of the algorithm. When solving large-scale nonlinear problems, state-of-the art IPM solvers rely on efficient sparse linear solvers to solve the KKT system. Instead, we propose a novel reduced-space IPM algorithm that condenses the KKT system into a dense matrix whose size is proportional to the number of degrees of freedom in the problem. Depending on where the reduction occurs we derive two variants of the reduced-space method: linearize-then-reduce and reduce-then-linearize. We adapt their workflow so that the vast majority of computations are accelerated on GPUs. We provide extensive numerical results on the optimal power flow problem, comparing our GPU-accelerated reduced space IPM with Knitro and a hybrid full space IPM algorithm. By evaluating the derivatives on the GPU and solving the KKT system on the CPU, the hybrid solution is already significantly faster than the CPU-only solutions. The two reduced-space algorithms go one step further by solving the KKT system entirely on the GPU. As expected, the performance of the two reduction algorithms depends intrinsically on the number of available degrees of freedom: their performance is poor when the problem has many degrees of freedom, but the two algorithms are up to 3 times faster than Knitro as soon as the relative number of degrees of freedom becomes smaller.Mihai Anitescu dedicates this work to the 70-th birthday of Florian Potra. Florian, thank you for the great contributions to optimization in general, and interior point methods in particular, and for initiating me and many others in them.

show abstract