Spectral Tensor-Train Decomposition

Abstract

Abstract. The accurate approximation of high-dimensional functions is an essential task in uncertainty quantification and many other fields. We propose a new function approximation scheme based on a spectral extension of the tensor-train (TT) decomposition. We first define a functional version of the TT decomposition and analyze its properties. We obtain results on the convergence of the decomposition, revealing links between the regularity of the function, the dimension of the input space, and the TT ranks. We also show that the regularity of the target function is preserved by the univariate functions (i.e., the "cores") comprising the functional TT decomposition. This result motivates an approximation scheme employing polynomial approximations of the cores. For functions with appropriate regularity, the resulting spectral tensor-train decomposition combines the favorable dimension-scaling of the TT decomposition with the spectral convergence rate of polynomial approximations, yielding efficient and accurate surrogates for high-dimensional functions. To construct these decompositions, we use the sampling algorithm TT-DMRG-cross to obtain the TT decomposition of tensors resulting from suitable discretizations of the target function. We assess the performance of the method on a range of numerical examples: a modifed set of Genz functions with dimension up to 100, and functions with mixed Fourier modes or with local features. We observe significant improvements in performance over an anisotropic adaptive Smolyak approach. The method is also used to approximate the solution of an elliptic PDE with random input data. The open source software and examples presented in this work are available online. 1 Key words. Approximation theory, tensor-train decomposition, orthogonal polynomials, uncertainty quantification. AMS subject classifications. 41A10, 41A63, 41A65, 46M05, 65D151. Introduction. High-dimensional functions appear frequently in science and engineering applications, where a quantity of interest may depend in nontrivial ways on a large number of independent variables. In the field of uncertainty quantification (UQ), for example, stochastic partial differential equations (PDEs) are often characterized by hundreds or thousands of independent stochastic parameters. A numerical approximation of the PDE solution must capture the coupled effects of all these parameters on the entire solution field, or on any quantity of interest that is a functional of the solution field. Problems of this kind quickly become intractable when confronted with naïve approximation methods, and the development of more effective methods is a long-standing challenge. This paper develops a new approach for high-dimensional function approximation, combining the discrete tensor-train format [49] with spectral theory for polynomial approximation.For simplicity, we will focus on real-valued functions representing the parameter dependence of a single quantity of interest. For a function f ∈ L 2 ([a, b] d ), a straightforward approximation might...

A continuous analogue of the tensor-train decomposition

“…The second method for obtaining a tensor is to represent a function in a tensor product basis, i.e., a basis produced by taking a full tensor product of univariate functions [13,31,6]. In particular, let {φ ki k : X k → R : k = 1, .…”

“…Parameters are represented by red squares and functions are represented in blue. In contrast to previous approaches [31,6], the parameters and parameterized form that represent each univariate function are stored separately, i.e., no tensor-product structure is imposed upon the parameters of the constituent univariate functions. Instead tensor-product structure is imposed upon the univariate functions themselves.…”

“…Tensor decompositions are a popular tool for mitigating the curse of dimensionality in applications that involve large-scale multiway arrays. Examples are wide-ranging and include compressing the results of scientific simulations [1], neural networks [2], numerical solution of PDEs [3], stochastic optimal control [4,5], and uncertainty quantification [6,7,8].…”

Reduced Order Modeling for Nonlinear PDE-constrained Optimization using Neural Networks

“…In ongoing work, we will, furthermore, explore different NN architectures such as residual networks, and reinforcement learning which has shown to be well-suited for time dependent data. Alternatively, instead of using NN other new approaches can be utilized to address the curse of dimensionality, such as spectral tensor train decompositions [25].…”

FFT-based homogenisation accelerated by low-rank tensor approximations

“…However, performing such operations with tensors in low-rank format, it typically happens that the representation rank of the tensors grows, which calls for their truncation, i.e. their approximation or reparametrisation with fewer parameters while keeping a reasonable accuracy [37,38,39]. This truncation of tensors may be viewed as a generalisation of the rounding of numbers, which occurs when working with floating point formats.…”