Shared-memory parallelization of MTTKRP for dense tensors

HayashiKoby,; BallardGrey,; JiangYujie,; TobiaMichael, J

doi:10.1145/3200691.3178522

Cited by 5 publications

(2 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The form of the derivative is a matricized tensor times Khatri-Rao product (MTTKRP) with the tensor Y and the Khatri-Rao product Z k . The MTTKRP is the dominant kernel in the standard CP computation in terms of computation time and has optimized high-performance implementations [7,56,40,31]. In the dense case, the MTTKRP costs O(rn d ).…”

Section: Gcp Gradientmentioning

confidence: 99%

Generalized Canonical Polyadic Tensor Decomposition

Hong¹,

Kolda²,

Duersch³

2020

SIAM Rev.

120

122

View full text Add to dashboard Cite

Tensor decomposition is a fundamental unsupervised machine learning method in data science, with applications including network analysis and sensor data processing. This work develops a generalized canonical polyadic (GCP) low-rank tensor decomposition that allows other loss functions besides squared error. For instance, we can use logistic loss or Kullback-Leibler divergence, enabling tensor decomposition for binary or count data. We present a variety of statisticallymotivated loss functions for various scenarios. We provide a generalized framework for computing gradients and handling missing data that enables the use of standard optimization methods for fitting the model. We demonstrate the flexibility of GCP on several real-world examples including interactions in a social network, neural activity in a mouse, and monthly rainfall measurements in India.

show abstract

Section: Gcp Gradientmentioning

confidence: 99%

Generalized Canonical Polyadic Tensor Decomposition

Hong¹,

Kolda²,

Duersch³

2020

SIAM Rev.

120

122

View full text Add to dashboard Cite

show abstract

“…Our proposed algorithm uses dimension trees, but we also benchmark our implementation without that optimization to highlight its importance. We use an existing implementation to perform the individual MTTKRPs [36] with this approach.…”

Section: Comparison Implementationsmentioning

confidence: 99%

Parallel Nonnegative CP Decomposition of Dense Tensors

Ballard

Hayashi

Kannan

2018

2018 IEEE 25th International Conference on High Performance Computing (HiPC)

Self Cite

View full text Add to dashboard Cite

The CP tensor decomposition is a low-rank approximation of a tensor. We present a distributed-memory parallel algorithm and implementation of an alternating optimization method for computing a CP decomposition of dense tensor data that can enforce nonnegativity of the computed low-rank factors. The principal task is to parallelize the matricized-tensor times Khatri-Rao product (MTTKRP) bottleneck subcomputation. The algorithm is computation efficient, using dimension trees to avoid redundant computation across MTTKRPs within the alternating method. Our approach is also communication efficient, using a data distribution and parallel algorithm across a multidimensional processor grid that can be tuned to minimize communication. We benchmark our software on synthetic as well as hyperspectral image and neuroscience dynamic functional connectivity data, demonstrating that our algorithm scales well to 100s of nodes (up to 4096 cores) and is faster and more general than the currently available parallel software.

show abstract

Accelerating alternating least squares for tensor decomposition by pairwise perturbation

Solomonik

2022

Numerical Linear Algebra App

View full text Add to dashboard Cite

The alternating least squares (ALS) algorithm for CP and Tucker decomposition is dominated in cost by the tensor contractions necessary to set up the quadratic optimization subproblems. We introduce a novel family of algorithms that uses perturbative corrections to the subproblems rather than recomputing the tensor contractions. This approximation is accurate when the factor matrices are changing little across iterations, which occurs when ALS approaches convergence. We provide a theoretical analysis to bound the approximation error.Our numerical experiments demonstrate that the proposed pairwise perturbation algorithms are easy to control and converge to minima that are as good as ALS. The experimental results show improvements of up to 3.1× with respect to state-of-the-art ALS approaches for various model tensor problems and real datasets.

show abstract

Shared-memory parallelization of MTTKRP for dense tensors

Cited by 5 publications

References 24 publications

Generalized Canonical Polyadic Tensor Decomposition

Generalized Canonical Polyadic Tensor Decomposition

Parallel Nonnegative CP Decomposition of Dense Tensors

Accelerating alternating least squares for tensor decomposition by pairwise perturbation

Contact Info

Product

Resources

About