Massively parallel methods for engineering and science problems

Camp, William J.; Plimpton, Steve; Hendrickson, Bruce; Leland, Robert W.

doi:10.1145/175276.175279

Cited by 42 publications

(22 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…These models can broadly be categorized as one-dimensional (1D) and two-dimensional (2D) partitioning models. In 1D models [30,31,[34][35][36][37][38][39] , each processor is responsible for a row/column stripe, whereas in 2D models, each processor may be responsible for a submatrix block (generally defined by a subset of rows and columns) or as in the most general case, each processor may be responsible for an arbitrarily defined subset of nonzeros. Compared to 1D models, 2D models possess more freedom in partitioning the coefficient matrix.…”

Section: Related Workmentioning

confidence: 99%

Reducing latency cost in 2D sparse matrix partitioning models

Selvitopi

Aykanat

2016

Parallel Computing

View full text Add to dashboard Cite

a b s t r a c tSparse matrix partitioning is a common technique used for improving performance of parallel linear iterative solvers. Compared to solvers used for symmetric linear systems, solvers for nonsymmetric systems offer more potential for addressing different multiple communication metrics due to the flexibility of adopting different partitions on the input and output vectors of sparse matrix-vector multiplication operations. In this regard, there exist works based on one-dimensional (1D) and two-dimensional (2D) fine-grain partitioning models that effectively address both bandwidth and latency costs in nonsymmetric solvers. In this work, we propose two new models based on 2D checkerboard and jagged partitioning. These models aim at minimizing total message count while maintaining a balance on communication volume loads of processors; hence, they address both bandwidth and latency costs. We evaluate all partitioning models on two nonsymmetric system solvers implemented using the widely adopted PETSc toolkit and conduct extensive experiments using these solvers on a modern system (a BlueGene/Q machine) successfully scaling them up to 8K processors. Along with the proposed models, we put practical aspects of eight evaluated models (two 1D-and six 2D-based) under thorough analysis. To the best of our knowledge, this is the first work that analyzes practical performance of 2D models on this scale. Among evaluated models, the models that rely on 2D jagged partitioning obtain the most promising results by striking a balance between minimizing bandwidth and latency costs.

show abstract

Section: Related Workmentioning

confidence: 99%

Reducing latency cost in 2D sparse matrix partitioning models

Selvitopi

Aykanat

2016

Parallel Computing

View full text Add to dashboard Cite

show abstract

“…The standard graph partitioning approach has been widely used to decompose irregular domains for the sake of efficient parallel execution [2,3,16,17,19,22,23,25,30,34]. In our previous works [6,7,9], we introduced two computational hypergraph models for the decomposition problems.…”

Section: Introductionmentioning

confidence: 99%

A hypergraph-partitioning approach for coarse-grain decomposition

Çatalyürek

Aykanat

2001

Proceedings of the 2001 ACM/IEEE Conference on Supercomputing

View full text Add to dashboard Cite

We propose a new two-phase method for the coarse-grain decomposition of irregular computational domains. This work focuses on the 2D partitioning of sparse matrices for parallel matrix-vector multiplication. However, the proposed model can also be used to decompose computational domains of other parallel reduction problems. This work also introduces the use of multi-constraint hypergraph partitioning, for solving the decomposition problem. The proposed method explicitly models the minimization of communication volume while enforcing the upper bound of p + q ; 2 on the maximum number of messages handled by a single processor, for a parallel system with P = p q processors. Experimental results on a wide range of realistic sparse matrices confirm the validity of the proposed methods, by achieving up to 25 percent better partitions than the standard graph model, in terms of total communication volume, and 59 percent better partitions in terms of number of messages, on the overall average.

show abstract

“…This is an extremely important issue because a large class of complex simulations used in industry today have irregular domains and/or dynamically changing interactions. For example, SPICE for circuit simulation, DYNA-3D and PRONTO-3D for structural mechanics modeling, GAUSSIAN and DMOL for quantum mechanical simulation of molecules, CHARMM and DISCOVER for molecular dynamics simulation of organic systems, and FIDAP for modeling complex fluid flows [8].…”

Section: Introductionmentioning

confidence: 99%

“…Thus, since the available parallelism in theses types of applications cannot be determined statically by present parallelizing compilers [6,8,11], compile-time analysis must be complemented by new methods capable of automatically extracting parallelism at run-time. The reason that run-time techniques are needed is that the access pattern of some programs cannot be determined statically, either because of limitations of the current analysis algorithms or because the access pattern is a function of the input data.…”

Section: Introductionmentioning

confidence: 99%

A scalable method for run-time loop parallelization

Rauchwerger

Amato

Padua³

1995

Int J Parallel Prog

View full text Add to dashboard Cite

Current parallelizing compilers do a reasonable job of extracting parallelism from programs with regular, well behaved, statically analyzable access patterns. However, they cannot extract a significant fraction of the available parallelism if the program has a complex and/or statically insufficiently defined access pattern, e.g., simulation programs with irregular domains and/or dynamically changing interactions. Since such programs represent a large fraction of all applications, techniques are needed for extracting their inherent parallelism at run-time. In this paper we give a new run-time technique for finding an optimal parallel execution schedule for a partially parallel loop, i.e., a loop whose parallelization requires synchronization to ensure that the iterations are executed in the correct order. Given the original loop, the compiler generates inspector code that performs run-time preprocessing of the loop's access pattern, and scheduler code that schedules (and executes) the loop iterations. The inspector is fully parallel, uses no synchronization, and can be applied to any loop (from which an inspector can be extracted). In addition, it can implement at run-time the two most effective transformations for increasing the amount of parallelism in a loop: array privatization and reduction parallelization (element-wise). The ability to identify privatizable and reduction variables is very powerful since it eliminates the data dependences involving these variables and thereby potentially increases the overall parallelism of the loop. We also describe a new scheme for constructing an optimal parallel execution schedule for the iterations of the loop. The schedule produced is a partition of the set of iterations into subsets called wavefronts so that there are no data dependences between iterations in a wavefront. Although the wavefronts themselves are constructed one after another, the computation of each wavefront is fully parallel and requires no synchronization. This new method has advantages over all previous run-time techniques for analyzing and scheduling partially parallel loops since none of them has all of these desirable properties.

show abstract

Massively parallel methods for engineering and science problems

Cited by 42 publications

References 8 publications

Reducing latency cost in 2D sparse matrix partitioning models

Reducing latency cost in 2D sparse matrix partitioning models

A hypergraph-partitioning approach for coarse-grain decomposition

A scalable method for run-time loop parallelization

Contact Info

Product

Resources

About