Low-rank separated representation surrogates of high-dimensional stochastic functions: Application in Bayesian inference

Abstract

This study introduces a non-intrusive approach in the context of low-rank separated representation to construct a surrogate of high-dimensional stochastic functions, e.g., PDEs/ODEs, in order to decrease the computational cost of Markov Chain Monte Carlo simulations in Bayesian inference. The surrogate model is constructed via a regularized alternative least-square regression with Tikhonov regularization using a roughening matrix computing the gradient of the solution, in conjunction with a perturbation-based error indicator to detect optimal model complexities. The model approximates a vector of a continuous solution at discrete values of a physical variable. The required number of random realizations to achieve a successful approximation linearly depends on the function dimensionality. The computational cost of the model construction is quadratic in the number of random inputs, which potentially tackles the curse of dimensionality in high-dimensional stochastic functions. Furthermore, this vector valued separated representation-based model, in comparison to the available scalar-valued case, leads to a significant reduction in the cost of approximation by an order of magnitude equal to the vector size. The performance of the method is studied through its application to three numerical examples including a 41-dimensional elliptic PDE and a 21-dimensional cavity flow.

Polynomial meta-models with canonical low-rank approximations: Numerical insights and comparison to sparse polynomial chaos expansions

“…Different algorithms have been proposed in the literature for building a decomposition in the form of Eq. ( 26) in a non-intrusive manner (Chevreuil et al, 2013;Doostan et al, 2013;Mathelin, 2014;Rai, 2014;Validi, 2014;Chevreuil et al, 2015). A common point is that the polynomial coefficients are determined by means of an alternated least-squares (ALS) minimization.…”

“…We conclude this section by briefly referring to alternative algorithms for developing LRA non-intrusively. The algorithms in Doostan et al (2013); Mathelin (2014); Validi (2014) involve a progressive increase of the rank as well. In Doostan et al (2013); Validi (2014), when the r-th rank-one component is added, the polynomial coefficients in the (r − 1) previously built rank-one terms are also updated.…”

“…More recently, canonical decompositions are attracting an increasing interest in the field of uncertainty quantification (Nouy, 2010;Chevreuil et al, 2013;Doostan et al, 2013;Hadigol et al, 2014;Mathelin, 2014;Rai, 2014;Validi, 2014;Chevreuil et al, 2015).…”

Sparse low-rank separated representation models for learning from data

“…Recently, it has been shown that imposing semi-orthogonality conditions on univariate factors ensures the existence of separated decompositions of tensors with order greater than 2 [3]. Low-rank separated representations based on ALS methods have been recently proposed for solving high-dimensional regression problems [8,9]. Tikhonov regularization procedures are used to circumvent overfitting, where the regularization parameters λ j (corresponding to the ALS subproblem in the jth dimension) are selected using generalized CV procedures that depend on the singular values of A j .…”

“…Tikhonov regularization procedures are used to circumvent overfitting, where the regularization parameters λ j (corresponding to the ALS subproblem in the jth dimension) are selected using generalized CV procedures that depend on the singular values of A j . Instead of using identity or diagonal regularization matrices as in [1], more general regularization matrices depending on the norm of the solution [8] or its gradient [9] have been derived. A perturbation-based error indicator is also used in [8] for selecting the separation rank and the number of basis functions used to approximate each component function.…”