SGD_Tucker: A Novel Stochastic Optimization Strategy for Parallel Sparse Tucker Decomposition

Li, Hao; Li, Zixuan; Li, Kenli; Rellermeyer, Jan S.; Chen, Lydia Y.; Li, Keqin

doi:10.48550/arxiv.2012.03550

Cited by 1 publication

(1 citation statement)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Kaya and Ucar [28] provide parallel algorithms for sparse Tucker decompositions. Li et al [32] introduce SGD-Tucker, which uses stochastic gradient descent to perform Tucker decomposition of sparse tensors.…”

Section: Previous Workmentioning

confidence: 99%

Fast and Accurate Randomized Algorithms for Low-rank Tensor Decompositions

Ma,

Solomonik

2021

Preprint

View full text Add to dashboard Cite

Low-rank Tucker and CP tensor decompositions are powerful tools in data analytics. The widely used alternating least squares (ALS) method, which solves a sequence of over-determined least squares subproblems, is inefficient for large and sparse tensors. We propose a fast and accurate sketched ALS algorithm for Tucker decomposition, which solves a sequence of sketched rank-constrained linear least squares subproblems. Theoretical sketch size upper bounds are provided to achieve O( )-relative error for each subproblem with two sketching techniques, TensorSketch and leverage score sampling. Experimental results show that this new ALS algorithm, combined with a new initialization scheme based on randomized range finder, yields up to 22.0% relative decomposition residual improvement compared to the state-of-the-art sketched randomized algorithm for Tucker decomposition of various synthetic datasets. This Tucker-ALS algorithm is further used to accelerate CP decomposition, by using randomized Tucker compression followed by CP decomposition of the Tucker core tensor. Experimental results show that this algorithm not only converges faster, but also yields more accurate CP decompositions.

show abstract

Section: Previous Workmentioning

confidence: 99%