Rank Bounds for Approximating Gaussian Densities in the Tensor-Train Format

Abstract

Low-rank tensor approximations have shown great potential for uncertainty quantification in high dimensions, for example, to build surrogate models that can be used to speed up largescale inference problems (Eigel et al., Inverse Problems 34, 2018; Dolgov et al., Statistics & Computing 30, 2020). The feasibility and efficiency of such approaches depends critically on the rank that is necessary to represent or approximate the underlying distribution. In this paper, a-priori rank bounds for approximations in the functional tensor-train representation for the case of Gaussian models are developed. It is shown that under suitable conditions on the precision matrix, the Gaussian density can be approximated to high accuracy without suffering from an exponential growth of complexity as the dimension increases. These results provide a rigorous justification of the suitability and the limitations of low-rank tensor methods in a simple but important model case. Numerical experiments confirm that the rank bounds capture the qualitative behavior of the rank structure when varying the parameters of the precision matrix and the accuracy of the approximation. Finally, the practical relevance of the theoretical results is demonstrated in the context of a Bayesian filtering problem.

Tensor-based Methods for Sequential State and Parameter Estimation in State Space Models

“…Some theoretical results exist that provide rank bounds. For example, the work of [24] establishes specific bounds for certain multivariate Gaussian densities that depend poly-logarithmically on m, while the work of [13] proves dimension-independent bounds for general functions in weighted spaces with dominating mixed smoothness.…”

Computing f‐divergences and distances of high‐dimensional probability density functions

“…We sketch three possibilities how to arrive at such a low-rank representation of the density. One is via tensor function representations 4,[8][9][10][11] of the pdf, 12,13 another via an analogous representation of the probabilistic characteristic function (pcf), [14][15][16] and a third one is via a low-rank representation of the RV itself-for example, References 17 and 18.…”

“…The first starting point is the assumption that one has or may obtain a low-rank tensor function representation 4,[8][9][10][11] of the pdf. 12,13 The second possible starting point in is that one has such a low-rank tensor function of the pcf. [14][15][16]22,23 Then via the Fourier transform one may come back to the pdf.…”

“…The goal is a discrete low‐rank tensor point evaluation of the pdf. The first starting point is the assumption that one has or may obtain a low‐rank tensor function representation 4,8‐11 of the pdf 12,13 . The second possible starting point in is that one has such a low‐rank tensor function of the pcf 14‐16,22,23 .…”

“…In a very recent paper, 13 the authors estimate tensor train (TT) ranks for approximated multivariate Gaussian pdfs. In Reference 12, the authors use the TT‐format to approximate multivariate probability distributions.…”