Model-Driven Sparse CP Decomposition for Higher-Order Tensors

Li, Jiajia; Choi, Jee; Perros, Ioakeim; Sun, Jimeng; Vuduc, Richard

doi:10.1109/ipdps.2017.80

Cited by 52 publications

(32 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, there has been growing interest in scaling tensor operations to bigger data and more processors in both the data mining/machine learning and the high performance computing communities. For sparse tensors, there have been parallelization efforts to compute CP decompositions both on shared-memory platforms [22], [23] as well as distributedmemory platforms [24]- [26], and these approaches can be generalized to constrained problems [13]. The focus of this work is on dense tensors, but many of the ideas for sparse tensors are applicable to the dense case, including parallel data distributions, communication pattern, and techniques to avoid recomputation across modes.…”

Section: Related Workmentioning

confidence: 99%

“…The idea of using dimension trees (discussed in section IV-A) to avoid recomputation within MTTKRPs across modes is introduced in [4] for computing the CP decomposition of dense tensors. It has also been used for sparse CP [23], [26] and other tensor computations [24].…”

Section: Related Workmentioning

confidence: 99%

“…An important optimization of the CP-ALS algorithm is to re-use temporary values across inner iterations [4], [23], [28], [29]. To illustrate the idea, consider a 3-way tensor X approximated by U, V, W and the two MTTKRP computations M (1) = X (1) (W V) and M (2) = X (2) (W U) used to update factor matrices U and V, respectively.…”

Section: A Dimension Treesmentioning

confidence: 99%

See 2 more Smart Citations

Parallel Nonnegative CP Decomposition of Dense Tensors

Ballard

Hayashi

Kannan

2018

2018 IEEE 25th International Conference on High Performance Computing (HiPC)

View full text Add to dashboard Cite

The CP tensor decomposition is a low-rank approximation of a tensor. We present a distributed-memory parallel algorithm and implementation of an alternating optimization method for computing a CP decomposition of dense tensor data that can enforce nonnegativity of the computed low-rank factors. The principal task is to parallelize the matricized-tensor times Khatri-Rao product (MTTKRP) bottleneck subcomputation. The algorithm is computation efficient, using dimension trees to avoid redundant computation across MTTKRPs within the alternating method. Our approach is also communication efficient, using a data distribution and parallel algorithm across a multidimensional processor grid that can be tuned to minimize communication. We benchmark our software on synthetic as well as hyperspectral image and neuroscience dynamic functional connectivity data, demonstrating that our algorithm scales well to 100s of nodes (up to 4096 cores) and is faster and more general than the currently available parallel software.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Parallel Nonnegative CP Decomposition of Dense Tensors

Ballard

Hayashi

Kannan

2018

2018 IEEE 25th International Conference on High Performance Computing (HiPC)

View full text Add to dashboard Cite

show abstract

“…Most of the available tensor analysis so ware packages [7,26] are wri en in Matlab, yielding limitations on performance and utility of multicore and other high-performance architectures. While there have been many recent developments in e cient soware for sparse tensor decompositions [15,22], there remain few options in the case of dense tensors, which is the subject of this work. Our motivating application is a neuroimaging data analysis problem involving functional MRI (fMRI) data.…”

Section: Introductionmentioning

confidence: 99%

Shared-memory parallelization of MTTKRP for dense tensors

et al. 2018

View full text Add to dashboard Cite

e matricized-tensor times Khatri-Rao product (MTTKRP) is the computational bo leneck for algorithms computing CP decompositions of tensors. In this paper, we develop shared-memory parallel algorithms for MTTKRP involving dense tensors. e algorithms cast nearly all of the computation as matrix operations in order to use optimized BLAS subroutines, and they avoid reordering tensor entries in memory. We benchmark sequential and parallel performance of our implementations, demonstrating high sequential performance and e cient parallel scaling. We use our parallel implementation to compute a CP decomposition of a neuroimaging data set and achieve a speedup of up to 7.4× over existing parallel so ware.

show abstract

“…Building on the CSF data structure, Li at al. proposed an adaptive tensor memoization algorithm that reduces the number of redundant floating-point operations that occur during the sequence of MTTKRP calculations required by CP-ALS, with the trade-off of increased memory usage [18] due to storing semi-sparse intermediate tensors. An adaptive model tuning framework called AdaTM was also developed that chooses an optimized memoization algorithm based on the sparse input tensor.…”

mentioning

confidence: 99%

Software for Sparse Tensor Decomposition on Emerging Computing Architectures

Phipps¹,

Kolda²

2019

SIAM J. Sci. Comput.

View full text Add to dashboard Cite

In this paper, we develop software for decomposing sparse tensors that is portable to and performant on a variety of multicore, manycore, and GPU computing architectures. The result is a single code whose performance matches optimized architecture-specific implementations.The key to a portable approach is to determine multiple levels of parallelism that can be mapped in different ways to different architectures, and we explain how to do this for the matricized tensor times Khatri-Rao product (MTTKRP) which is the key kernel in canonical polyadic tensor decomposition. Our implementation leverages the Kokkos framework, which enables a single code to achieve high performance across multiple architectures that differ in how they approach fine-grained parallelism. We also introduce a new construct for portable thread-local arrays, which we call compile-time polymorphic arrays. Not only are the specifics of our approaches and implementation interesting for tuning tensor computations, but they also provide a roadmap for developing other portable highperformance codes. As a last step in optimizing performance, we modify the MTTKRP algorithm itself to do a permuted traversal of tensor nonzeros to reduce atomic-write contention. We test the performance of our implementation on 16-and 68-core Intel CPUs and the K80 and P100 NVIDIA GPUs, showing that we are competitive with state-of-the-art architecture-specific codes while having the advantage of being able to run on a variety of architectures.

show abstract

Model-Driven Sparse CP Decomposition for Higher-Order Tensors

Cited by 52 publications

References 19 publications

Parallel Nonnegative CP Decomposition of Dense Tensors

Parallel Nonnegative CP Decomposition of Dense Tensors

Shared-memory parallelization of MTTKRP for dense tensors

Software for Sparse Tensor Decomposition on Emerging Computing Architectures

Contact Info

Product

Resources

About