Accelerating sparse matrix computations via data compression

Willcock, Jeremiah; Lumsdaine, Andrew

doi:10.1145/1183401.1183444

Cited by 115 publications

(74 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This is the first advantage our implementation conferred on the Cell. Note that our technique is less general but simpler than a recent index compression approach [23].…”

Section: Index Size Selectionmentioning

confidence: 99%

Optimization of sparse matrix–vector multiplication on emerging multicore platforms

et al. 2009

View full text Add to dashboard Cite

We are witnessing a dramatic change in computer architecture due to the multicore paradigm shift, as every electronic device from cell phones to supercomputers confronts parallelism of unprecedented scale. To fully unleash the potential of these systems, the HPC community must develop multicore specific optimization methodologies for important scientific computations. In this work, we examine sparse matrix-vector multiply (SpMV) -one of the most heavily used kernels in scientific computing -across a broad spectrum of multicore designs. Our experimental platform includes the homogeneous AMD dual-core and Intel quad-core designs, the heterogeneous STI Cell, as well as the first scientific study of the highly multithreaded Sun Niagara2. We present several optimization strategies especially effective for the multicore environment, and demonstrate significant performance improvements compared to existing state-of-the-art serial and parallel SpMV implementations. Additionally, we present key insights into the architectural tradeoffs of leading multicore design strategies, in the context of demanding memory-bound numerical algorithms.

show abstract

“…This is the first advantage our implementation conferred on the Cell. Note that our technique is less general but simpler than a recent index compression approach [23].…”

Section: Index Size Selectionmentioning

confidence: 99%

Optimization of sparse matrix–vector multiplication on emerging multicore platforms

et al. 2009

View full text Add to dashboard Cite

show abstract

“…These patterns include blocks [13], variable or mixtures of differently-sized blocks [12] diagonals, which may be especially wellsuited to machines with SIMD and vector units [32,28], general pattern compression [33], value compression [15], and combinations.…”

Section: Related Workmentioning

confidence: 99%

Reduced-Bandwidth Multithreaded Algorithms for Sparse Matrix-Vector Multiplication

Buluç

Williams

Oliker

et al. 2011

2011 IEEE International Parallel &Amp; Distributed Processing Symposium

109

View full text Add to dashboard Cite

Abstract-On multicore architectures, the ratio of peak memory bandwidth to peak floating-point performance (byte:flop ratio) is decreasing as core counts increase, further limiting the performance of bandwidth limited applications. Multiplying a sparse matrix (as well as its transpose in the unsymmetric case) with a dense vector is the core of sparse iterative methods. In this paper, we present a new multithreaded algorithm for the symmetric case which potentially cuts the bandwidth requirements in half while exposing lots of parallelism in practice. We also give a new data structure transformation, called bitmasked register blocks, which promises significant reductions on bandwidth requirements by reducing the number of indexing elements without introducing additional fill-in zeros. Our work shows how to incorporate this transformation into existing parallel algorithms (both symmetric and unsymmetric) without limiting their parallel scalability. Experimental results indicate that the combined benefits of bitmasked register blocks and the new symmetric algorithm can be as high as a factor of 3.5x in multicore performance over an already scalable parallel approach. We also provide a model that accurately predicts the performance of the new methods, showing that even larger performance gains are expected in future multicore systems as current trends (decreasing byte:flop ratio and larger sparse matrices) continue.

show abstract

“…Many approaches have been suggested to reduce the memory bandwidth requirements in SpMV: row/column reordering [38,37], register blocking [41], compressing row or column indices [45] , cache blocking [25,46], symmetry [39], using single or mixed precision [16], and reorganizing the SpMV ordering across multiple iterations in a solver [35], among others. Some of these approaches are hard to parallelize.…”

Section: Introductionmentioning

confidence: 99%

“…The indices are normally represented as integers, but there are various ways to reduce their size. Willcok and Lumsdaine [45] apply graph compression techniques to reduce the size, showing speedups of up to 33% (although much more modest numbers on average). Williams et al point out that by using cache blocking, it is possible to reduce the number of bits for the column indices since the number of columns in the block is typically small [46].…”

Section: Introductionmentioning

confidence: 99%

Hierarchical Diagonal Blocking and Precision Reduction Applied to Combinatorial Multigrid

Blelloch

Koutis

Tangwongsan

2010

2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis

View full text Add to dashboard Cite

Abstract-Memory bandwidth is a major limiting factor in the scalability of parallel iterative algorithms that rely on sparse matrix-vector multiplication (SpMV). This paper introduces Hierarchical Diagonal Blocking (HDB), an approach which we believe captures many of the existing optimization techniques for SpMV in a common representation. Using this representation in conjuction with precision-reduction techniques, we develop and evaluate high-performance SpMV kernels. We also study the implications of using our SpMV kernels in a complete iterative solver. Our method of choice is a Combinatorial Multigrid solver that can fully utilize our fastest reduced-precision SpMV kernel without sacrificing the quality of the solution. We provide extensive empirical evaluation of the effectiveness of the approach on a variety of benchmark matrices, demonstrating substantial speedups on all matrices considered.

show abstract

Accelerating sparse matrix computations via data compression

Cited by 115 publications

References 19 publications

Optimization of sparse matrix–vector multiplication on emerging multicore platforms

Optimization of sparse matrix–vector multiplication on emerging multicore platforms

Reduced-Bandwidth Multithreaded Algorithms for Sparse Matrix-Vector Multiplication

Hierarchical Diagonal Blocking and Precision Reduction Applied to Combinatorial Multigrid

Contact Info

Product

Resources

About