Multivariate Regression and Machine Learning with Sums of Separable Functions

Recently, the format of TT tensors (Hackbusch and Kühn in J Fourier Anal has turned out to be a promising new format for the approximation of solutions of high dimensional problems. In this paper, we prove some new results for the TT representation of a tensor U ∈ R n 1 ×···×n d and for the manifold of tensors of TT-rank r . As a first result, we prove that the TT (or compression) ranks r i of a tensor U are unique and equal to the respective separation ranks of U if the components of the TT decomposition are required to fulfil a certain maximal rank condition. We then show that the set T of TT tensors of fixed rank r locally forms an embedded manifold in R n 1 ×···×n d , therefore preserving the essential theoretical properties of the Tucker format, but often showing an improved scaling behaviour. Extending a similar approach for matrices (Conte and Lubich in M2AN 44:759, 2010), we introduce certain gauge conditions to obtain a unique representation of the tangent space T U T of T and deduce a local parametrization of the TT manifold. The parametrisation of T U T is often crucial for an algorithmic treatment of high-dimensional time-dependent PDEs and minimisation problems (Lubich in From quantum to classical molecular dynamics: reduced methods and numerical analysis, 2008). We conclude with remarks on those applications and present some numerical examples.

show abstract

“…We use a variation of the Friedman1 data set [14] for example used in [3]. For d = 5 and n i = n with n ∈ {3, .…”

Section: Numerical Examplesmentioning

confidence: 99%

On manifolds of tensors of fixed TT-rank

2011

View full text Add to dashboard Cite

show abstract

“…The problems of multidimensional scattered data modeling and data mining are known to lead to computationally intensive simulations. We refer to [12,31,9,23,28] for discussion of the most commonly used computational approaches in this field of numerical analysis.…”

Section: Multidimensional Data Modelingmentioning

confidence: 99%

Range-Separated Tensor Format for Many-Particle Modeling

Benner

Khoromskaia

Khoromskij

2018

SIAM J. Sci. Comput.

View full text Add to dashboard Cite

Abstract. We introduce and analyze the new range-separated (RS) canonical/Tucker tensor format which aims for numerical modeling of the 3D long-range interaction potentials in multiparticle systems. The main idea of the RS tensor format is the independent grid-based low-rank representation of the localized and global parts in the target tensor, which allows the efficient numerical approximation of N -particle interaction potentials. The single-particle reference potential, described by the radial basis function p( x ), x ∈ R d , say p( x ) = 1/ x for d = 3, is split into a sum of localized and long-range low-rank canonical tensors represented on a fine 3D n×n×n Cartesian grid. The smoothed long-range contribution to the total potential sum is represented on the 3D grid in O(n) storage via the low-rank canonical/Tucker tensor. We prove that the Tucker rank parameters depend only logarithmically on the number of particles N and the grid size n. Agglomeration of the short-range part in the sum is reduced to an independent treatment of N localized terms with almost disjoint effective supports, calculated in O(N ) operations. Thus, the cumulated sum of shortrange clusters is parametrized by a single low-rank canonical reference tensor with local support, accomplished by a list of particle coordinates and their charges. The RS canonical/Tucker tensor representations defined on the fine n × n × n Cartesian grid reduce the cost of multilinear algebraic operations on the 3D potential sums, arising in modeling of the multidimensional data by radial basis functions. For instance, computation of the electrostatic potential of a large biomolecule and the interaction energy of a many-particle system, 3D integration and convolution transforms as well as low-parametric fitting of multidimensional scattered data all are reduced to 1D calculations.

show abstract

“…A class of low tensor rank preconditioners for the multidimensional elliptic problems with jumping coefficients in R d is proposed in [18]. The employment of separation of variables in machine learning is addressed in [6].…”

Section: On Tensor-structured Solution Of Multidimensional Equationsmentioning

confidence: 99%

Tensors-structured numerical methods in scientific computing: Survey on recent advances

Khoromskij¹

2012

Chemometrics and Intelligent Laboratory Systems

161

153

View full text Add to dashboard Cite

In the present paper, we give a survey of the recent results and outline future prospects of the tensor-structured numerical methods in applications to multidimensional problems in scientific computing. The guiding principle of the tensor methods is an approximation of multivariate functions and operators relying on certain separation of variables. Along with the traditional canonical and Tucker models, we focus on the recent quantics-TT tensor approximation method that allows to represent N -d tensors with log-volume complexity, O(d log N ). We outline how these methods can be applied in the framework of tensor truncated iteration for the solution of the high-dimensional elliptic/parabolic equations and parametric PDEs. Numerical examples demonstrate that the tensor-structured methods have proved their value in application to various computational problems arising in quantum chemistry and in the multi-dimensional/parametric FEM/BEM modeling-the tool apparently works and gives the promise for future use in challenging high-dimensional applications.

show abstract

Multivariate Regression and Machine Learning with Sums of Separable Functions

Cited by 97 publications

References 23 publications

On manifolds of tensors of fixed TT-rank

On manifolds of tensors of fixed TT-rank

Range-Separated Tensor Format for Many-Particle Modeling

Tensors-structured numerical methods in scientific computing: Survey on recent advances

Contact Info

Product

Resources

About