Scalable Tucker Factorization for Sparse Tensors - Algorithms and Discoveries

Abstract

Given sparse multi-dimensional data (e.g., (user, movie, time; rating) for movie recommendations), how can we discover latent concepts/relations and predict missing values? Tucker factorization has been widely used to solve such problems with multi-dimensional data, which are modeled as tensors. However, most Tucker factorization algorithms regard and estimate missing entries as zeros, which triggers a highly inaccurate decomposition. Moreover, few methods focusing on an accuracy exhibit limited scalability since they require huge memory and heavy computational costs while updating factor matrices.In this paper, we propose P-TUCKER, a scalable Tucker factorization method for sparse tensors. P-TUCKER performs alternating least squares with a row-wise update rule in a fully parallel way, which significantly reduces memory requirements for updating factor matrices. Furthermore, we offer two variants of P-TUCKER: a caching algorithm P-TUCKER-CACHE and an approximation algorithm P-TUCKER-APPROX, both of which accelerate the update process. Experimental results show that P-TUCKER exhibits 1.7-14.1× speed-up and 1.4-4.8× less error compared to the state-of-the-art. In addition, P-TUCKER scales near linearly with the number of observable entries in a tensor and number of threads. Thanks to P-TUCKER, we successfully discover hidden concepts and relations in a large-scale real-world tensor, while existing methods cannot reveal latent features due to their limited scalability or low accuracy.

Randomized Algorithms for Computation of Tucker Decomposition and Higher Order SVD (HOSVD)

“…Recently, several scalable algorithms were proposed in [116]- [118] for computation of the HOSVD. These algorithms inherently do not have a randomized structure and basically they exploit the idea of on-the-fly-computation and parallel row-wise update rule roles to avoid the intermediate data explosion problem.…”

A Unified Fourth-Order Tensor-Based Smart Community System

“…For example, a high-order tensor usually has enormous elements, which require high computing capability to process data. Efficient tensor decomposition algorithms have been recently proposed in [27][28][29], which can achieve up to 14 times faster tensor decomposition. How to further accelerate tensor decomposition and reduce tensor data size as far as possible is still an open problem, which deserves significant efforts in future work.…”

Gtensor

“…• Gtensor uses a row-wise update rule [10,11] Thanks to its careful design, Gtensor outperforms existing tensor analysis library in terms of running time and accuracy, as we show in Section 2.1.2.…”

“…• Tensor decomposition: PARAFAC, nonnegative PARAFAC, Tucker [5,10,11], nonnegative Tucker, and CMTF (Coupled Matrix-Tensor Factorization) [3,6]. • Tensor generation: generating 1) tensors filled with one or random values, 2) tensors from given factor matrices and a core tensor, and 3) sparse R-MAT and Kronecker tensors.…”