Iteration-fusing conjugate gradient

Zhuang, Sicong; Casas, Marc

doi:10.1145/3079079.3079091

Cited by 14 publications

(25 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since relation (43) with k = 0 states that z (1) j+1 + σ 0 v j = Av j , expression (47) very closely resembles the finite precision Lanczos recurrence relation (38). In particular, we point out that the matrix G −1 j+1 does not occur in the recursive update (47) for v j+1 . We clarify the difference between the finite precision and exact variant of recurrence relation (47) in Section 4.4.…”

Section: Derivation Of a Stable Pipelined Cg Methodsmentioning

confidence: 77%

See 1 more Smart Citation

Numerically Stable Recurrence Relations for the Communication Hiding Pipelined Conjugate Gradient Method

Cools

Cornelis

Vanroose

2019

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

Pipelined Krylov subspace methods (also referred to as communication-hiding methods) have been proposed in the literature as a scalable alternative to classic Krylov subspace algorithms for iteratively computing the solution to a large linear system in parallel. For symmetric and positive definite system matrices the pipelined Conjugate Gradient method, p(l)-CG, outperforms its classic Conjugate Gradient counterpart on large scale distributed memory hardware by overlapping global communication with essential computations like the matrix-vector product, thus "hiding" global communication. A well-known drawback of the pipelining technique is the (possibly significant) loss of numerical stability. In this work a numerically stable variant of the pipelined Conjugate Gradient algorithm is presented that avoids the propagation of local rounding errors in the finite precision recurrence relations that construct the Krylov subspace basis. The multi-term recurrence relation for the basis vector is replaced by three-term recurrences, improving stability without increasing the overall computational cost of the algorithm. The proposed modification ensures that the pipelined Conjugate Gradient method is able to attain a highly accurate solution independently of the pipeline length. Numerical experiments demonstrate a combination of excellent parallel performance and improved maximal attainable accuracy for the new pipelined Conjugate Gradient algorithm. This work thus resolves one of the major practical restrictions for the useability of pipelined Krylov subspace methods.

show abstract

Section: Derivation Of a Stable Pipelined Cg Methodsmentioning

confidence: 77%

“…The final recurrence relation to update z (9), is replaced by the (stable) relation (47). This update explicitly uses the auxiliary variable z (1) i−2 and implicitly depends on the other l − 1 = 2 auxiliary variables z (2) i−2 and z (3) i−2 through the respective recurrence relations.…”

Section: Examplementioning

confidence: 99%

Numerically Stable Recurrence Relations for the Communication Hiding Pipelined Conjugate Gradient Method

Cools

Cornelis

Vanroose

2019

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

show abstract

“…This approach is based on the code developed in [28], where an improved version of the conjugate gradient method is presented.…”

Section: Blockingmentioning

confidence: 99%

Towards an Auto-Tuned and Task-Based SpMV (LASs Library)

Catalán

Usui

Toledo

et al. 2020

OpenMP: Portable Multi-Level Parallelism on Modern Systems

View full text Add to dashboard Cite

We present a novel approach to parallelize the SpMV kernel included in LASs (Linear Algebra routines on OmpSs) library, after a deep review and analysis of several well-known approaches. LASs is based on OmpSs, a task-based runtime that extends OpenMP directives, providing more flexibility to apply new strategies. Based on tasking and nesting, with the aim of improving the workload imbalance inherent to the SpMV operation, we present a strategy especially useful for highly imbalanced input matrices. In this approach, the number of created tasks is dynamically decided in order to maximize the use of the resources of the platform. Throughout this paper, SpMV behavior depending on the selected strategy (state of the art and proposed strategies) is deeply analyzed, setting in this way the base for a future auto-tunable code that is able to select the most suitable approach depending on the input matrix. The experiments of this work were carried out for a set of 12 matrices from the Suite Sparse Matrix Collection, all of them with different characteristics regarding their sparsity. The experiments of this work were performed on a node of Marenostrum 4 supercomputer (with two sockets Intel Xeon, 24 cores each) and on a node of Dibona cluster (using one ARM ThunderX2 socket with 32 cores). Our tests show that, for Intel Xeon, the best parallelization strategy reduces the execution time of the reference MKL multi-threaded version up to 67%. On ARM ThunderX2, the reduction is up to 56% with respect to the OmpSs parallel reference.

show abstract

“…Over the last decades various approaches on reducing or eliminating the synchronization bottleneck in Krylov subspace methods have been proposed [64,10,9,17,20,26,18,12]. Recent methods that aim to eliminate global synchronization points include improved Krylov subspace methods [73,72,74], hierarchical Krylov subspace methods [46], enlarged Krylov subspace methods [39], an iteration fusing Conjugate Gradient method [75], s-step Krylov subspace methods [11,43,6,5,4,44], and pipelined Krylov subspace methods [31,32,57,24,71]. Pipelined Krylov subspace methods aim to avoid communication latency by reducing the number of global synchronization bottlenecks and by hiding global communication behind useful computational work.…”

Section: Introductionmentioning

confidence: 99%

Analyzing and improving maximal attainable accuracy in the communication hiding pipelined BiCGStab method

Cools

2019

Parallel Computing

View full text Add to dashboard Cite

Pipelined Krylov subspace methods avoid communication latency by reducing the number of global synchronization bottlenecks and by hiding global communication behind useful computational work. In exact arithmetic pipelined Krylov subspace algorithms are equivalent to classic Krylov subspace methods and generate identical series of iterates. However, as a consequence of the reformulation of the algorithm to improve parallelism, pipelined methods may suffer from severely reduced attainable accuracy in a practical finite precision setting. This work presents a numerical stability analysis that describes and quantifies the impact of local rounding error propagation on the maximal attainable accuracy of the multi-term recurrences in the preconditioned pipelined BiCGStab method. Theoretical expressions for the gaps between the true and computed residual as well as other auxiliary variables used in the algorithm are derived, and the elementary dependencies between the gaps on the various recursively computed vector variables are analyzed. The norms of the corresponding propagation matrices and vectors provide insights in the possible amplification of local rounding errors throughout the algorithm. Stability of the pipelined BiCGStab method is compared numerically to that of pipelined CG on a symmetric benchmark problem. Furthermore, numerical evidence supporting the effectiveness of employing a residual replacement type strategy to improve the maximal attainable accuracy for the pipelined BiCGStab method is provided.

show abstract

Iteration-fusing conjugate gradient

Cited by 14 publications

References 23 publications

Numerically Stable Recurrence Relations for the Communication Hiding Pipelined Conjugate Gradient Method

Numerically Stable Recurrence Relations for the Communication Hiding Pipelined Conjugate Gradient Method

Towards an Auto-Tuned and Task-Based SpMV (LASs Library)

Analyzing and improving maximal attainable accuracy in the communication hiding pipelined BiCGStab method

Contact Info

Product

Resources

About