A flexible class of parallel matrix multiplication algorithms

Gunnels, John A.; Lin, Chih‐Hsun; Morrow, Greg; Geijn, Robert van de

doi:10.1109/ipps.1998.669898

Cited by 41 publications

(37 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Interestingly enough, the shapes discussed are exactly those that we encountered when studying a class of matrix multiplication algorithms on distributed memory architectures [5]. This is not surprising, since distributed memory is just another layer in the memory hierarchy.…”

Section: Discussionsupporting

confidence: 58%

A Family of High-Performance Matrix Multiplication Algorithms

Gunnels

Henry

Geijn

2006

Applied Parallel Computing. State of the Art in Scientific Computing

View full text Add to dashboard Cite

Abstract. During the last half-decade, a number of research efforts have centered around developing software for generating automatically tuned matrix multiplication kernels. These include the PHiPAC project and the ATLAS project. The software endproducts of both projects employ brute force to search a parameter space for blockings that accommodate multiple levels of memory hierarchy. We take a different approach: using a simple model of hierarchical memories we employ mathematics to determine a locally-optimal strategy for blocking matrices. The theoretical results show that, depending on the shape of the matrices involved, different strategies are locally-optimal. Rather than determining a blocking strategy at library generation time, the theoretical results show that, ideally, one should pursue a heuristic that allows the blocking strategy to be determined dynamically at run-time as a function of the shapes of the operands. When the resulting family of algorithms is combined with a highly optimized inner-kernel for a small matrix multiplication, the approach yields performance that is superior to that of methods that automatically tune such kernels. Preliminary results, for the Intel Pentium (R) III processor, support the theoretical insights.

show abstract

Section: Discussionsupporting

confidence: 58%

A Family of High-Performance Matrix Multiplication Algorithms

Gunnels

Henry

Geijn

2006

Applied Parallel Computing. State of the Art in Scientific Computing

View full text Add to dashboard Cite

show abstract

“…In this way, minimal coding effort is required to attain superior performance across a wide spectrum of algorithms and problem sizes. Analogously, in [10] we demonstrate that an efficient implementation of parallel matrix multiplication requires both multiple algorithms and a method for selecting the appropriate algorithm for the presented case if one is to handle operands of various sizes and shapes. In [16] we came to a similar conclusion in the context of out-of-core factorization algorithms.…”

Section: So Many Algorithms So Little Timementioning

confidence: 76%

Formal Methods for High-Performance Linear Algebra Libraries

Gunnels

Geijn

2001

IFIP Advances in Information and Communication Technology

View full text Add to dashboard Cite

A colleague of ours, Dr. Timothy Mattson of Intel, once made the following observation: "Literature professors read literature. Computer Science professors should at least occasionally read code." The point he was making was that in order to write superior prose one needs to read good (and bad) literature. Analogously, it is our thesis that exposure to elegant (and ugly) programs tends to yield the insights which are necessary if one wishes to produce consistently well-written code.Since the advent of high-performance distributed-memory parallel computing, the need for intelligible code has become ever greater. Development and maintenance of libraries for these kinds of architectures is simply too complex to be amenable to conventional approaches to coding. Attempting to do so has led to the production of an abundance of inefficient, anfractuous code that is difficult to maintain and nigh-impossible to upgrade.Having struggled with these issues for more than a decade, we have arrived at a conclusion which is somewhat surprising to us: the answer is to apply formal methods from Computer Science to the development of high-performance linear algebra libraries. The resulting approach has consistently resulted in aesthetically-pleasing, coherent code that greatly facilitates performance analysis, intelligent modularity, and the enforcement of program correctness via assertions. Since the technique is completely language-independent, it lends itself equally well to a wide spectrum of programming languages (and paradigms) ranging from C and Fortran to C++ and Java. In this paper, we illustrate our observations by looking at our Formal Linear Algebra Methods Environment (FLAME).

show abstract

“…As discussed above, matrix inversion operation can be parallelized. Similarly, there are well-known classical algorithms for parallelizing matrix multiplication (Gunnels et al, 1998).…”

Section: Parallelization Opportunitiesmentioning

confidence: 99%

Locality-sensitive and Re-use Promoting Personalized PageRank computations

2015

View full text Add to dashboard Cite

Abstract:Node distance/proximity measures are used for quantifying how nearby or otherwise related two or more nodes on a graph are. In particular, personalized PageRank (PPR) based measures of node proximity have been shown to be highly effective in many prediction and recommendation applications. Despite its effectiveness, however, the use of personalized PageRank for large graphs is difficult due to its high computation cost. In this paper, we propose a Locality-sensitive, Re-use promoting, approximate Personalized PageRank (LR-PPR) algorithm for efficiently computing the PPR values relying on the localities of the given seed nodes on the graph: (a) The LR-PPR algorithm is locality sensitive in the sense that it reduces the computational cost of the PPR computation process by focusing on the local neighborhoods of the seed nodes.(b) LR-PPR is re-use promoting in that instead of performing a monolithic computation for the given seed node set using the entire graph, LR-PPR divides the work into localities of the seeds and caches the intermediary results obtained during the computation. These cached results are then reused for future queries sharing seed nodes. Experiment results for different data sets and under different scenarios show that LR-PPR algorithm is highly-efficient and accurate. Abstract. Node distance/proximity measures are used for quantifying how nearby or otherwise related two or more nodes on a graph are. In particular, personalized PageRank (PPR) based measures of node proximity have been shown to be highly effective in many prediction and recommendation applications. Despite its effectiveness, however, the use of personalized PageRank for large graphs is difficult due to its high computation cost. In this paper, we propose a Localitysensitive, Re-use promoting, approximate Personalized PageRank (LR-PPR) algorithm for efficiently computing the PPR values relying on the localities of the given seed nodes on the graph: (a) The LR-PPR algorithm is locality sensitive in the sense that it reduces the computational cost of the PPR computation process by focusing on the local neighborhoods of the seed nodes. (b) LR-PPR is re-use promoting in that instead of performing a monolithic computation for the given seed node set using the entire graph, LR-PPR divides the work into localities of the seeds and caches the intermediary results obtained during the computation. These cached results are then reused for future queries sharing seed nodes. Experiment results for different data sets and under different scenarios show that LR-PPR algorithm is highly-efficient and accurate. Powered by Editorial Manager® and ProduXion Manager® from Aries Systems Corporation

show abstract

A flexible class of parallel matrix multiplication algorithms

Cited by 41 publications

References 11 publications

A Family of High-Performance Matrix Multiplication Algorithms

A Family of High-Performance Matrix Multiplication Algorithms

Formal Methods for High-Performance Linear Algebra Libraries

Locality-sensitive and Re-use Promoting Personalized PageRank computations

Contact Info

Product

Resources

About