Low-rank tensor reconstruction of concentrated densities with application to Bayesian inversion

Abstract

Transport maps have become a popular mechanic to express complicated probability densities using sample propagation through an optimized pushforward. Beside their broad applicability and well-known success, transport maps suffer from several drawbacks such as numerical inaccuracies induced by the optimization process and the fact that sampling schemes have to be employed when quantities of interest, e.g. moments are to compute. This paper presents a novel method for the accurate functional approximation of probability density functions (PDF) that copes with those issues. By interpreting the pull-back result of a target PDF through an inexact transport map as a perturbed reference density, a subsequent functional representation in a more accessible format allows for efficient and more accurate computation of the desired quantities. We introduce a layer-based approximation of the perturbed reference density in an appropriate coordinate system to split the high-dimensional representation problem into a set of independent approximations for which separately chosen orthonormal basis functions are available. This effectively motivates the notion of h-and p-refinement (i.e. "mesh size" and polynomial degree) for the approximation of high-dimensional PDFs. To circumvent the curse of dimensionality and enable sampling-free access to certain quantities of interest, a low-rank reconstruction in the tensor train format is employed via the Variational Monte Carlo method. An a priori convergence analysis of the developed approach is derived in terms of Hellinger distance and the Kullback-Leibler divergence. Applications comprising Bayesian inverse problems and several degrees of concentrated densities illuminate the (superior) convergence in comparison to Monte Carlo and Markov-Chain Monte Carlo methods.

Efficient approximation of high-dimensional exponentials by tensornetworks

“…Usually, again sampling methods are used for this often high-dimensional problem, the most popular of which certainly is the Markov chain Monte Carlo method. Nevertheless, recently some developments took place which showed that functional approximations of (posterior) densities are feasible and may prove beneficial in terms of convergence rates [11,48], see also [25,49] for different low-rank techniques.…”

“…In UQ this task arises for example in the parameter reconstruction of model data via inverse problems [7][8][9]. If a functional representation of exppℓpy; δqq can be constructed, it may for instance be used to efficiently generate independent posterior samples [10] or to compute highdimensional quantities of interest such as moments or marginals [11].…”

“…This work focuses in particular on the tensor train (TT) format [14], also know as matrix product states, as a technique to compress the high-dimensional coefficient tensor c. Given such a representation of the diffusion coefficient κ it is easy to apply the SG algorithm [15]. If the likelihood function exppℓpy; δqq is given in TT format quantities of interest [11] can be computed promptly and samples can be draw efficiently [10].…”

“…These approaches allow for a fast generation of independent samples from the posterior, but generally do not supply a functional representation thereof. Recent results to gain such a functional representation in TT format are presented in [10,11], which is motivated by the low-rank representability of Gaussian densities [25]. However, the approximation task itself can be quite challenging since one needs to guarantee small Lipschitz constants of the transport map [26] to achieve numerical stability.…”

Non-intrusive Tensor Reconstruction for High-Dimensional Random PDEs

“…Some details on different tensor formats and numerical algorithms can be found in [30][31][32][33]. Our focus lies on the TT format [33], which has been used with tremendous success for the solution of parametric PDEs and related UQ problems such as Bayesian inversion and random field representations, see [34][35][36][37][38]. ASGFEM in hierachical tensor formats are presented in [2,39] for affine coefficients.…”