An Algebraic Parallel Treecode in Arbitrary Dimensions

March, William B.; Xiao, Bo; Yu, Chih‐Wei; Biros, George

doi:10.1109/ipdps.2015.86

Cited by 14 publications

(32 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…MPI-GOFMM is parallelized using MPI+OpenMP, following the algorithms for distributed-memory treecodes described in [18] and the task-based runtime scheduler design in [1]. The latter will be generalized to perform out-of-order execution in the distributed memory setting.…”

Section: B Distributed-memory Parallelismmentioning

confidence: 99%

Distributed-Memory Hierarchical Compression of Dense SPD Matrices

Reiz

Biros

2018

SC18: International Conference for High Performance Computing, Networking, Storage and Analysis

Self Cite

View full text Add to dashboard Cite

We present a distributed memory algorithm for the hierarchical compression of symmetric positive definite (SPD) matrices. Our method is based on GOFMM, an algorithm that appeared in doi:10.1145/3126908.3126921. For many SPD matrices, GOFMM enables compression and approximate matrix-vector multiplication that for many matrices can reach N log N time-as opposed to N 2 required for a dense matrix. But GOFMM supports only shared memory parallelism. In this paper, we use the message passing interface (MPI) and extend the ideas of GOFMM to the distributed memory setting. We also propose and implement an asynchronous algorithm for faster multiplication. We present different usage scenarios on a selection of SPD matrices that are related to graphs, neural-networks, and covariance operators. We present results on the Texas Advanced Computing Center's "Stampede 2" system. We also compare with the STRUMPACK software package, which, to our knowledge, is the only other available software that can compress arbitrary SPD matrices in parallel. In our largest run, we were able to compress a 67M-by-67M matrix in less than three minutes and perform a multiplication with 512 vectors within 5 seconds on 6,144 Intel "Skylake" cores.

show abstract

Section: B Distributed-memory Parallelismmentioning

confidence: 99%

Distributed-Memory Hierarchical Compression of Dense SPD Matrices

Reiz

Biros

2018

SC18: International Conference for High Performance Computing, Networking, Storage and Analysis

Self Cite

View full text Add to dashboard Cite

show abstract

“…In [24], we presented GSKS (General Stride Kernel Summation), an matrix-free kernel summation that performs fusing optimization. While the best-known method computes…”

Section: Fast Kernel Summationmentioning

confidence: 99%

An N log N Parallel Fast Direct Solver for Kernel Matrices

March

Biros

2017

2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS)

Self Cite

View full text Add to dashboard Cite

Kernel matrices appear in machine learning and non-parametric statistics. Given N points in d dimensions and a kernel function that requires O(d) work to evaluate, we present an O(dN log N )-work algorithm for the approximate factorization of a regularized kernel matrix, a common computational bottleneck in the training phase of a learning task. With this factorization, solving a linear system with a kernel matrix can be done with O(N log N ) work. Our algorithm only requires kernel evaluations and does not require that the kernel matrix admits an efficient global low rank approximation. Instead our factorization only assumes low-rank properties for the off-diagonal blocks under an appropriate row and column ordering. We also present a hybrid method that, when the factorization is prohibitively expensive, combines a partial factorization with iterative methods. As a highlight, we are able to approximately factorize a dense 11M × 11M kernel matrix in 2 minutes on 3,072 x86 "Haswell" cores and a 4.5M × 4.5M matrix in 1 minute using 4,352 "Knights Landing" cores.

show abstract

“…In reality, this number differs for each tree node and it is chosen adaptively so that the approximation error is below a tolerance τ . Finally, the parallelization and scalability of ASKIT was described in [25]. Another version of ASKIT that resembles the relationship between fast-multipole methods and treecodes is described in [23].…”

Section: B Askitmentioning

confidence: 99%

INV-ASKIT: A Parallel Fast Direct Solver for Kernel Matrices

March

Xiao

et al. 2016

2016 IEEE International Parallel and Distributed Processing Symposium (IPDPS)

Self Cite

View full text Add to dashboard Cite

We present a parallel algorithm for computing the approximate factorization of an N -by-N kernel matrix. Once this factorization has been constructed (with N log 2 N work), we can solve linear systems with this matrix with N log N work. Kernel matrices represent pairwise interactions of points in metric spaces. They appear in machine learning, approximation theory, and computational physics. Kernel matrices are typically dense (matrix multiplication scales quadratically with N ) and ill-conditioned (solves can require 100s of Krylov iterations). Thus, fast algorithms for matrix multiplication and factorization are critical for scalability.Recently we introduced ASKIT, a new method for approximating a kernel matrix that resembles N-body methods.Here we introduce INV-ASKIT, a factorization scheme based on ASKIT. We describe the new method, derive complexity estimates, and conduct an empirical study of its accuracy and scalability. We report results on realworld datasets including "COVTYPE" (0.5M points in 54 dimensions), "SUSY" (4.5M points in 8 dimensions) and "MNIST" (2M points in 784 dimensions) using shared and distributed memory parallelism. In our largest run we approximately factorize a dense matrix of size 32M × 32M (generated from points in 64 dimensions) on 4,096 Sandy-Bridge cores. To our knowledge these results improve the state of the art by several orders of magnitude.

show abstract

An Algebraic Parallel Treecode in Arbitrary Dimensions

Cited by 14 publications

References 12 publications

Distributed-Memory Hierarchical Compression of Dense SPD Matrices

Distributed-Memory Hierarchical Compression of Dense SPD Matrices

An N log N Parallel Fast Direct Solver for Kernel Matrices

INV-ASKIT: A Parallel Fast Direct Solver for Kernel Matrices

Contact Info

Product

Resources

About