Stable ALS approximation in the TT-format for rank-adaptive tensor completion

Grasedyck, Lars; Krämer, Sebastian

doi:10.1007/s00211-019-01072-4

Cited by 27 publications

(27 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When |v| y = |v(y)| is used, u M,n is known as the nonlinear least squares estimator of u. The extensive interest in machine learning in recent years has lead to the investigation of this estimator for special model classes like sparse vectors [3,7,8], low-rank tensors [5,[9][10][11][12] and neural networks [13,14]. However, to the knowledge of the authors no investigation for general model classes has been published so far.…”

Section: Convergence Bounds For Empirical Nonlinear Least-squaresmentioning

confidence: 99%

Convergence bounds for empirical nonlinear least-squares

2022

View full text Add to dashboard Cite

We consider best approximation problems in a nonlinear subset [[EQUATION]] of a Banach space of functions [[EQUATION]] . The norm is assumed to be a generalization of the [[EQUATION]] -norm for which only a weighted Monte Carlo estimate [[EQUATION]] can be computed. The objective is to obtain an approximation [[EQUATION]] of an unknown function [[EQUATION]] by minimizing the empirical norm [[EQUATION]] . We consider this problem for general nonlinear subsets and establish error bounds for the empirical best approximation error. Our results are based on a restricted isometry property (RIP) which holds in probability and is independent of the nonlinear least squares setting. Several model classes are examined where analytical statements can be made about the RIP and the results are compared to existing sample complexity bounds from the literature. We find that for well-studied model classes our general bound is weaker but exhibits many of the same properties as these specialized bounds. Notably, we demonstrate the advantage of an optimal sampling density (as known for linear spaces) for sets of functions with sparse representations.

show abstract

Section: Convergence Bounds For Empirical Nonlinear Least-squaresmentioning

confidence: 99%

Convergence bounds for empirical nonlinear least-squares

2022

View full text Add to dashboard Cite

show abstract

“…Although, not directly suitable for recovery tasks, it became apparent that DMRG and ALS can be adapted to work in this context. Two of these extensions to the ALS algorithm are the stablilized ALS approximation (SALSA) [21] and the block alternating steepest descent for Recovery (bASD) algorithm [13]. Both adapt the tensor network ranks and are better suited to the problem of data identification.…”

Section: Introductionmentioning

confidence: 99%

“…It is possible to include rank adaptivity as in SALSA[21] or bASD[13] and we have noted this in the relevant places.2 The orthogonality comes from the symmetry of L which results in orthogonal eigenspaces.Frontiers in Applied Mathematics and Statistics | www.frontiersin.org September 2021 | Volume 7 | Article 702486…”

mentioning

confidence: 99%

A Block-Sparse Tensor Train Format for Sample-Efficient High-Dimensional Polynomial Regression

Götte

Schneider

Trunschke

2021

Front. Appl. Math. Stat.

View full text Add to dashboard Cite

Low-rank tensors are an established framework for the parametrization of multivariate polynomials. We propose to extend this framework by including the concept of block-sparsity to efficiently parametrize homogeneous, multivariate polynomials with low-rank tensors. This provides a representation of general multivariate polynomials as a sum of homogeneous, multivariate polynomials, represented by block-sparse, low-rank tensors. We show that this sum can be concisely represented by a single block-sparse, low-rank tensor.We further prove cases, where low-rank tensors are particularly well suited by showing that for banded symmetric tensors of homogeneous polynomials the block sizes in the block-sparse multivariate polynomial space can be bounded independent of the number of variables.We showcase this format by applying it to high-dimensional least squares regression problems where it demonstrates improved computational resource utilization and sample efficiency.

show abstract

“…Tensor completion has been widely studied in the literature. Various approaches like decomposition-based methods [29][30][31][32][33][34] are available in the literature. Various other approaches can be found in other works.…”

Section: Introductionmentioning

confidence: 99%

Quantized CP approximation and sparse tensor interpolation of function‐generated data

Khoromskij

Naraparaju

Schneider

2019

Numerical Linear Algebra App

View full text Add to dashboard Cite

Summary In this article, we consider the iterative schemes to compute the canonical polyadic (CP) approximation of quantized data generated by a function discretized on a large uniform grid in an interval on the real line. This paper continues the research on the quantics‐tensor train (QTT) method (“O(d log N)‐quantics approximation of N‐d tensors in high‐dimensional numerical modeling” in Constructive Approximation, 2011) developed for the tensor train (TT) approximation of the quantized images of function related data. In the QTT approach, the target vector of length 2L is reshaped to a Lth‐order tensor with two entries in each mode (quantized representation) and then approximated by the QTT tensor including 2r2L parameters, where r is the maximal TT rank. In what follows, we consider the alternating least squares (ALS) iterative scheme to compute the rank‐r CP approximation of the quantized vectors, which requires only 2rL≪2L parameters for storage. In the earlier papers (“Tensors‐structured numerical methods in scientific computing: survey on recent advances” in Chemom Intell Lab Syst, 2012), such a representation was called QCan format, whereas in this paper, we abbreviate it as the QCP (quantized canonical polyadic) representation. We test the ALS algorithm to calculate the QCP approximation on various functions, and in all cases, we observed the exponential error decay in the QCP rank. The main idea for recovering a discretized function in the rank‐r QCP format using the reduced number of the functional samples, calculated only at O(2rL) grid points, is presented. The special version of the ALS scheme for solving the arising minimization problem is described. This approach can be viewed as the sparse QCP‐interpolation method that allows to recover all 2rL representation parameters of the rank‐r QCP tensor. Numerical examples show the efficiency of the QCP‐ALS‐type iteration and indicate the exponential convergence rate in r.

show abstract

Stable ALS approximation in the TT-format for rank-adaptive tensor completion

Cited by 27 publications

References 42 publications

Convergence bounds for empirical nonlinear least-squares

Convergence bounds for empirical nonlinear least-squares

A Block-Sparse Tensor Train Format for Sample-Efficient High-Dimensional Polynomial Regression

Quantized CP approximation and sparse tensor interpolation of function‐generated data

Contact Info

Product

Resources

About