A New Truncation Strategy for the Higher-Order Singular Value Decomposition

Vannieuwenhoven, Nick; Vandebril, Raf; Meerbergen, Karl

doi:10.1137/110836067

Cited by 250 publications

(259 citation statements)

References 51 publications

Supporting

Mentioning

258

Contrasting

Order By: Relevance

“…Introduction. Data-sparse representations of elements living in the tensor product of (finite dimensional) vector spaces have become an intensely studied subject in recent years, yielding a myriad of representations, such as the higher-order singular value decomposition [23,55,56], CANDECOMP/PARAFAC decomposition (CPD) [15,31], Block-term decompositions [22], H-Tucker [28,30] and Tensor Trains [44], each with varying assumptions and divergent applications. Among these, the CPD is the oldest; according to [11], its roots (for symmetric tensors) can be traced back to algebraic geometry in the middle of 19th century as featured in the work of Sylvester.…”

mentioning

confidence: 99%

On Generic Nonexistence of the Schmidt--Eckart--Young Decomposition for Complex Tensors

Vannieuwenhoven¹,

Nicaise²,

Vandebril³

et al. 2014

SIAM J. Matrix Anal. & Appl.

View full text Add to dashboard Cite

The Schmidt-Eckart-Young theorem for matrices states that the optimal rank-r approximation to a matrix is obtained by retaining the first r terms from the singular value decomposition of that matrix. This work considers a generalization of this optimal truncation property to the CANDECOMP/PARAFAC decomposition of tensors and establishes a necessary orthogonality condition. We prove that this condition is not satisfied at least by an open set of positive Lebesgue measure in complex tensor spaces. It is proved moreover that for complex tensors of small rank this condition can only be satisfied by a set of tensors of Lebesgue measure zero.Keywords : Canonical Polyadic decomposition, CANDECOMP, PARAFAC, orthogonal rank decomposition, Tensor singular value decomposition, EckartYoung theorem MSC : Primary : 14A10, 14A25, 14Q15, 15A21, 15A69, 15B99. ON GENERIC NONEXISTENCE OF THE SCHMIDT-ECKART-YOUNG DECOMPOSITION FOR COMPLEX TENSORSN. VANNIEUWENHOVEN † , J. NICAISE ‡ , R. VANDEBRIL † , AND K. MEERBERGEN † Abstract. The Schmidt-Eckart-Young theorem for matrices states that the optimal rank-r approximation to a matrix is obtained by retaining the first r terms from the singular value decomposition of that matrix. This work considers a generalization of this optimal truncation property to the CANDECOMP/PARAFAC decomposition of tensors and establishes a necessary orthogonality condition. We prove that this condition is not satisfied at least by an open set of positive Lebesgue measure in complex tensor spaces. It is proved moreover that for complex tensors of small rank this condition can only be satisfied by a set of tensors of Lebesgue measure zero.

show abstract

mentioning

confidence: 99%

On Generic Nonexistence of the Schmidt--Eckart--Young Decomposition for Complex Tensors

Vannieuwenhoven¹,

Nicaise²,

Vandebril³

et al. 2014

SIAM J. Matrix Anal. & Appl.

View full text Add to dashboard Cite

show abstract

“…In particular, for mrank-(1, R, R) approximations it was shown to perform at least as well as THOSVD. Simulation results performed in [11] suggest that this superiority holds in most cases. For the particular case of rank-one approximations, [12] has come up with a two-stage algorithm called sequential rank-one approximation and projection (SeROAP), which first reduces dimensionality just like SeMP and then performs a sequence of projections for refining the approximation.…”

Section: Introductionmentioning

confidence: 87%

“…Though it is suboptimal, its error is bounded by a multiple of the optimal value. The alternative proposed in [11], which we refer to as sequentially optimal modal projections (SeMP), proceeds similarly but computes the modal projectors in a sequential fashion. This leads to the same error bound, but at a smaller cost due to the reduced size of the SVDs.…”

Section: Introductionmentioning

confidence: 99%

A novel non-iterative algorithm for low-multilinear-rank tensor approximation

Goulart

Comon

2017

2017 25th European Signal Processing Conference (EUSIPCO)

View full text Add to dashboard Cite

Abstract-Low-rank tensor approximation algorithms are building blocks in tensor methods for signal processing. In particular, approximations of low multilinear rank (mrank) are of central importance in tensor subspace analysis. This paper proposes a novel non-iterative algorithm for computing a low-mrank approximation, termed sequential low-rank approximation and projection (SeLRAP). Our algorithm generalizes sequential rankone approximation and projection (SeROAP), which aims at the rank-one case. For third-order mrank-(1,R,R) approximations, SeLRAP's outputs are always at least as accurate as those of previously proposed methods. Our simulation results suggest that this is actually the case for the overwhelmingly majority of random third-and fourth-order tensors and several different mranks. Though the accuracy improvement is often small, we show it can make a large difference when repeatedly computing approximations, as happens, e.g., in an iterative hard thresholding algorithm for tensor completion.

show abstract

“…Typical 3D Tucker compression approaches involve knowing in advance the desired core tensor size, so that either a) a rank-R 1 R 2 R 3 approximation can be directly computed [16]; or b) a subset of an existing approximation is selected in favor of a lower memory usage, and at the cost of reduced reconstruction quality. Setting the target ranks R i beforehand simplifies the problem and can be used for a faster decomposition (Vannieuwenhoven et al [28]). However, selecting a meaningful number of ranks is a nontrivial issue [12].…”

Section: Related Workmentioning

confidence: 99%

Lossy volume compression using Tucker truncation and thresholding

Ballester-Ripoll

Pajarola

2015

Vis Comput

View full text Add to dashboard Cite

Tensor decompositions, in particular the Tucker model, are a powerful family of techniques for dimensionality reduction and are being increasingly used for compactly encoding large multidimensional arrays, images and other visual data sets. In interactive applications, volume data often needs to be decompressed and manipulated dynamically; when designing data reduction and reconstruction methods, several parameters must be taken into account, such as the achievable compression ratio, approximation error and reconstruction speed. Weighing these variables in an effective way is challenging, and here we present two main contributions to solve this issue for Tucker tensor decompositions. First, we provide algorithms to efficiently compute, store and retrieve good choices of tensor rank selection and decompression parameters in order to optimize memory usage, approximation quality and computational costs. Second, we propose a Tucker compression alternative based on coefficient thresholding and zigzag traversal, followed by logarithmic quantization on both the transformed tensor core and its factor matrices. In terms of approximation accuracy, this approach is theoretically and empirically better than the commonly used tensor rank truncation method. Abstract Tensor decompositions, in particular the Tucker model, are a powerful family of techniques for dimensionality reduction and are being increasingly used for compactly encoding large multidimensional arrays, images and other visual data sets. In interactive applications, volume data often needs to be decompressed and manipulated dynamically; when designing data reduction and reconstruction methods, several parameters must be taken into account, such as the achievable compression ratio, approximation error and reconstruction speed. Weighing these variables in an effective way is challenging, and here we present two main contributions to solve this issue for Tucker tensor decompositions. First, we provide algorithms to efficiently compute, store and retrieve good choices of tensor rank selection and decompression parameters in order to optimize memory usage, approximation quality and computational costs. Second, we propose a Tucker compression alternative based on coefficient thresholding and zigzag traversal, followed by logarithmic quantization on both the transformed tensor core and its factor matrices. In terms of approximation accuracy, this approach is theoretically and empirically better than the commonly used tensor rank truncation method.

show abstract

A New Truncation Strategy for the Higher-Order Singular Value Decomposition

Cited by 250 publications

References 51 publications

On Generic Nonexistence of the Schmidt--Eckart--Young Decomposition for Complex Tensors

On Generic Nonexistence of the Schmidt--Eckart--Young Decomposition for Complex Tensors

A novel non-iterative algorithm for low-multilinear-rank tensor approximation

Lossy volume compression using Tucker truncation and thresholding

Contact Info

Product

Resources

About