Density Estimation in Uncertainty Propagation Problems Using a Surrogate Model

Abstract

The effect of uncertainties and noise on a quantity of interest (model output) is often better described by its probability density function (PDF) than by its moments. Although density estimation is a common task, the adequacy of approximation methods (surrogate models) for density estimation has not been analyzed before in the uncertainty-quantification (UQ) literature. In this paper, we first show that standard surrogate models (such as generalized polynomial chaos), which are highly accurate for moment estimation, might completely fail to approximate the PDF, even for one-dimensional noise. This is because density estimation requires that the surrogate model accurately approximates the gradient of the quantity of interest, and not just the quantity of interest itself. Hence, we develop a novel spline-based algorithm for density-estimation whose convergence rate in L q is polynomial in the sampling resolution. This convergence rate is better than that of standard statistical density-estimation methods (such as histograms and kernel density estimators) at dimensions 1 ≤ d ≤ 5 2 m, where m is the spline order. Furthermore, we obtain the convergence rate for density estimation with any surrogate model that approximates the quantity of interest and its gradient in L ∞ . Finally, we demonstrate our algorithm for problems in nonlinear optics and fluid dynamics.1 g in L q , the corresponding density p g might not be a good approximation of p f . Indeed, for p g to approximate p f , then g ′ needs to be close to f ′ , and g ′ (α) should also vanish if and only if f ′ (α) does. These conditions might not be satisfied by some of the above-mentioned standard UQ methods. In contrast, spline interpolation approximates both the function and its gradient [3,25,41,45], and does not tend to produce spurious extremal points. Therefore, we construct a novel algorithm for density estimation in uncertainty-propagation problems using splines as our surrogate model. With cubic splines, our density-estimation algorithm has a guaranteed convergence rate of at least h 3 , where h is the maximal sampling spacing (resolution). More generally, with splines of order m, the convergence rate is at least h m . These rates are superior to those of the standard kernel density estimators [52,59], for noise dimension 1 ≤ d ≤ 5 2 m. Our choice of splines is motivated by the availability and efficiency of one-and multi-dimensional spline toolboxes. Nevertheless, other surrogate models can be used in this algorithm, and indeed this paper lays the theoretical framework for deriving the convergence rate of such methods (Corollaries 4.8 and 5.5). We show, essentially, that density estimation convergence can be performed with any surrogate model for which the L ∞ error of both the function and its gradient converge to zero as the spacing resolution h vanishes. Because we only rely on solving the underlying deterministic model (i.e., our method is non-intrusive), and because interpolation by spline is a standard numerical procedure, our proposed method is...

The Wasserstein Distances Between Pushed-Forward Measures with Applications to Uncertainty Quantification

“…This in turn immediately implies an upper bound on the L 1 distance between the CDFs, due to the previously-noted Salvemini-Vallender identity W 1 (µ, ν) = F µ − F ν 1 [49]. The upper bounds on the Wasserstein-error stand in sharp contrast to the L q errors between the PDFs, since in general an upper bound on f − g q does not guarantee an upper bound on p µ −p ν L p , for any finite p and q [14]. We therefore see that the way we define the approximationerror in this problem is not a mere technicality, but rather determines the results of the convergence analysis.…”

“…Moreover, since the theorem requires that |(f * ) ′ |, |(g * ) ′ | > τ : > 0, we can assume without loss of generality that f and g are strongly monotonically decreasing. Next, we have the following standard lemma (for proof, see e.g., [14]):…”

“…Consider the case where ̺ is the Lebesgue measure on [0, 1] . Then, since both functions are monotonic, p µ (y) = 1/f ′ (f −1 (y)) = 1 and p ν (y) = 1/g ′ (g −1 (y)), see [14] for details. Hence, p ν is onto [1.1 −1 , 0.9 −1 ] ≈ [0.91, 1.11] and so p µ − p ν ∞ > 0.1, irrespectively of δ, see Fig.…”

“…Instead, we only have an approximation for the quantities of interest at our disposal. Indeed, the general settings presented above have spurred numerous papers in a field of computational science known as Uncertainty-Quantification (UQ), see e.g., [14,22,43,53,54,55]. Perhaps surprisingly, the full approximation of µ (rather than its moments alone) in these particular settings received little theoretical attention in the literature, even though it is of practical importance in diverse fields such as ocean waves [1], computational fluid dynamics [8], hydrology [9], aeronautics [17], biochemistry [25], and nonlinear optics [30,36].…”

“…Admittedly, the L p distances between the PDFs are both natural in practice and are associated with rich statistical theory; for p = 1, then p µ − p ν 1 is twice the total variation [13], and p µ − p ν 2 2 is the Integrated Square Error, which is a building block in non-parametric statistics [47]. Nevertheless, the analysis of the norms p µ − p ν p in terms of the functions f and g can be technically cumbersome; if e.g., ̺ is the Lebesgue measure, then p µ (y) is proportional to f −1 (y) 1/|∇f | dσ, where dσ is the (d − 1) dimensional surface measure [14]. Moreover, the distance p µ − p ν p is difficult to work with since it assumes that µ and ν have distributions.…”