Block and asynchronous two-stage methods for mildly nonlinear systems

Bai, Zhong‐Zhi; Migallón, Violeta; Penadés, José; Szyld, Daniel B.

doi:10.1007/s002110050409

Cited by 39 publications

(26 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…1.9], which is identical to (4). Note that even though CARP1 already appeared in [11] (without any specific name), no proof of convergence was given there; naturally, the convergence properties of CARP1 follow from Theorem 1.…”

Section: Carp1: Blocks Of One Equationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

“…In the second mode of operation, sometimes referred to as block-parallel, the blocks are assigned to different processors to be processed in parallel, and the results from all the blocks are then combined in some manner to produce the next iterate. Typical examples of block-parallel schemes are parallel versions of , RGMRES [30], conjugate gradient (CG), and CG-squared (CGS) [31]; other examples include those in [2,4,9,12,13].Kaczmarz's row projection method (KACZ) [23] was one of the first iterative methods used for large nonsymmetric systems. Its main advantages are robustness, guaranteed convergence on consistent systems, and cyclic convergence on inconsistent systems [8,16,32].…”

mentioning

confidence: 99%

See 2 more Smart Citations

Component-Averaged Row Projections: A Robust, Block-Parallel Scheme for Sparse Linear Systems

Gordon¹,

Gordon²

2005

SIAM J. Sci. Comput.

View full text Add to dashboard Cite

Abstract.A new method for the parallel solution of large sparse linear systems is introduced. It proceeds by dividing the equations into blocks and operating in block-parallel iterative mode; i.e., all the blocks are processed in parallel, and the partial results are "merged" to form the next iterate. The new scheme performs Kaczmarz row projections within the blocks and merges the results by certain component-averaging operations-hence it is called component-averaged row projections, or CARP. The system matrix can be general, nonsymmetric, and ill-conditioned, and the division into blocks is unrestricted. For partial differential equations (PDEs), if the blocks are domain-based, then only variables at the boundaries between domains are averaged, thereby minimizing data transfer between processors. CARP is very robust; its application to test cases of linear systems derived from PDEs shows that it converges in difficult cases where state-of-the-art methods fail. It is also very memory efficient and exhibits an almost linear speedup ratio, with efficiency greater than unity in some cases. A formal proof of convergence is presented: It is shown that the component-averaging operations are equivalent to row projections in a certain superspace, so the convergence properties of CARP are identical to those of Kaczmarz's algorithm in the superspace. CARP and its convergence proof also apply to the consistent convex feasibility problem. 1. Introduction. Iterative methods for solving large sparse linear systems of equations are advantageous over the classical direct solvers, especially for huge systems. Methods of parallelizing iterative algorithms [20,29] divide the equations into blocks and usually fall into one of two modes of operation. In the block-sequential (also called block-iterative) mode, the blocks are processed sequentially, but the computations on each block's equations are done in parallel. Examples of block-sequential methods are found in [1,6,10]. In the second mode of operation, sometimes referred to as block-parallel, the blocks are assigned to different processors to be processed in parallel, and the results from all the blocks are then combined in some manner to produce the next iterate. Typical examples of block-parallel schemes are parallel versions of , RGMRES [30], conjugate gradient (CG), and CG-squared (CGS) [31]; other examples include those in [2,4,9,12,13].Kaczmarz's row projection method (KACZ) [23] was one of the first iterative methods used for large nonsymmetric systems. Its main advantages are robustness, guaranteed convergence on consistent systems, and cyclic convergence on inconsistent systems [8,16,32]. KACZ was also independently discovered in the context of image reconstruction from projections, where it is called ART (algebraic reconstruction technique) [21]. Kaczmarz's algorithm, by its nature and mathematical definition, is inherently sequential since, at each iterative step, the current iterate is projected

show abstract

Section: Carp1: Blocks Of One Equationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Component-Averaged Row Projections: A Robust, Block-Parallel Scheme for Sparse Linear Systems

Gordon¹,

Gordon²

2005

SIAM J. Sci. Comput.

View full text Add to dashboard Cite

show abstract

“…It should also be mentioned that the shift function may not be the same in different thread blocks. While the GPU hardware encourages this approach, the idea is similar to a two-staged asynchronous iteration [7]. Mixed Precision Iterative Refinement.…”

Section: Mathematical Backgroundmentioning

confidence: 99%

GPU-Accelerated Asynchronous Error Correction for Mixed Precision Iterative Refinement

Antz

Łuszczek

Dongarra

et al. 2011

View full text Add to dashboard Cite

Abstract. In hardware-aware high performance computing, block-asynchronous iteration and mixed precision iterative refinement are two techniques that may be used to leverage the computing power of SIMD accelerators like GPUs in the iterative solution of linear equation systems. Although they use a very different approach for this purpose, they share the basic idea of compensating the convergence properties of an inferior numerical algorithm by a more efficient usage of the provided computing power. In this paper, we analyze the potential of combining both techniques. Therefore, we derive a mixed precision iterative refinement algorithm using a block-asynchronous iteration as an error correction solver, and compare its performance with a pure implementation of a blockasynchronous iteration and an iterative refinement method using double precision for the error correction solver. For matrices from the University of Florida Matrix collection, we report the convergence behaviour and provide the total solver runtime using different GPU architectures.

show abstract

“…The nonlinear system of form (1.1) has been investigated extensively owing to various scientific and engineering applications [1,3,7,10,23,24]. Normally, it can't get the exactly solutions even when n is very small, which promotes greatly the substantial developments of constructing various kinds of iterative methods.…”

Section: Introductionmentioning

confidence: 99%