Scalable conditional deep inverse Rosenblatt transports using tensor-trains and gradient-based dimension reduction

Abstract

We present a novel offline-online method to mitigate the computational burden of the characterization of conditional beliefs in statistical learning. In the offline phase, the proposed method learns the joint law of the belief random variables and the observational random variables in the tensor-train (TT) format. In the online phase, it utilizes the resulting order-preserving conditional transport map to issue real-time characterization of the conditional beliefs given new observed information. Compared with the state-of-the-art normalizing flows techniques, the proposed method relies on function approximation and is equipped with thorough performance analysis. This also allows us to further extend the capability of transport maps in challenging problems with high-dimensional observations and high-dimensional belief variables. On the one hand, we present novel heuristics to reorder and/or reparametrize the variables to enhance the approximation power of TT. On the other, we integrate the TT-based transport maps and the parameter reordering/reparametrization into layered compositions to further improve the performance of the resulting transport maps. We demonstrate the efficiency of the proposed method on various statistical learning tasks in ordinary differential equations (ODEs) and partial differential equations (PDEs).Preprint. Under review.

Self-reinforced polynomial approximation methods for concentrated probability densities

“…The computational cost is invariant to variable ordering. This is more flexible than the squared-tensor-train methods of [14,15], where marginalizing variables in arbitrary order can significantly increase the computational complexity.…”

“…Instead of directly operating with the high-dimensional discretized parameter θ, we apply the data-free dimension reduction method of [15,16] to compress the parameter dimensionality. We first construct a sensitivity matrix H ∈ R d θ ×d θ by integrating the Fisher information matrix over the prior distribution, and then compute the eigenpairs (ϕ i , ν i )-in descending order of eigenvalues ν i -of the matrix pencil (H, C −1 ).…”

“…The idea is to compute the exact KR rearrangement of an approximate density of the target random variable. For instance, tensor methods are used in [14,15,21] to construct the approximate density. By doing so, this approach bypasses the highly nonlinear optimization procedure, and the associated error analysis is often well established.…”

“…In this paper, we present generalizations and improvements of the function approximation methods used to build the KR rearrangement from density. Instead of using tensor methods as in the seminal papers [14,15], we propose in Section 2 to approximate the target density using a sparse tensorproduct basis via a least square formulation. More specifically, we approximate the square root of the unnormalized target density to ensure the nonnegativity of the density approximation by squaring the resulting function.…”

Low-rank Wasserstein polynomial chaos expansions in the framework of optimal transport

“…is typical for Markov chain Monte Carlo methods. With this in mind, functional representations in a polynomial basis exploiting the beneficial structure of the Knothe-Rosenblatt transform were for instance developed in [74,21,20]. For the formulation of the variational problem, the Kullback-Leibler divergence is used.…”