Output-weighted optimal sampling for Bayesian regression and rare event statistics using few samples

Abstract

<jats:p>For many important problems the quantity of interest is an unknown function of the parameters, which is a random vector with known statistics. Since the dependence of the output on this random vector is unknown, the challenge is to identify its statistics, using the minimum number of function evaluations. This problem can be seen in the context of active learning or optimal experimental design. We employ Bayesian regression to represent the derived model uncertainty due to finite and small number of input–output pairs. In this context we evaluate existing methods for optimal sample selection, such as model error minimization and mutual information maximization. We show that for the case of known output variance, the commonly employed criteria in the literature do not take into account the output values of the existing input–output pairs, while for the case of unknown output variance this dependence can be very weak. We introduce a criterion that takes into account the values of the output for the existing samples and adaptively selects inputs from regions of the parameter space which have an important contribution to the output. The new method allows for application to high-dimensional inputs, paving the way for optimal experimental design in high dimensions.</jats:p>

Output-Weighted Optimal Sampling for Bayesian Experimental Design and Uncertainty Quantification

“…When the likelihood ratio appears in an acquisition function, we refer to the latter as a ``likelihood-weighted"" (LW) acquisition function. To the best of our knowledge, the only LW acquisition function that has been proposed for BED is the so-called Q criterion [43]:…”

“…In goal-oriented uncertainty quantification, the choice of acquisition function largely depends on the features of the quantity of interest one wishes to identify (e.g., the statistical expectation of the output [38], the tails of the output distribution [34], or the maximum value that the black-box function can produce [49]). Acquisition functions come in various shapes and forms [8,45], but many popular criteria suffer from severe limitations, including high computational cost, intractability in high dimensions, and inability to discriminate between active and idle input variables [43].…”

“…We introduce a class of acquisition functions for active learning of black-box functions specifically designed for systems capable of generating extreme events. As in [34] and [43], the proposed criteria guide the search algorithm toward regions of the input space that are associated with unusual output values associated with rare events. Unlike [34], their computational complexity is comparable with that of traditional active-learning approaches, and they can be approximated in a way that makes the approach tractable in high dimensions.…”

“…Unlike [34], their computational complexity is comparable with that of traditional active-learning approaches, and they can be approximated in a way that makes the approach tractable in high dimensions. Unlike [43], they are derived for a much larger class of models (i.e., nonlinear, nonparametric models). In addition, the proposed criteria contain a mechanism that accounts for the importance of the output relative to the input and therefore are not limited to rare-event quantification but can be applied to any problem related to experimental design and uncertainty quantification.…”

Explaining People’s Worry Levels During the Covid-19 Pandemic: An Analysis of Socio-Economic and Cultural Dimensions

“…The rationale for choosing Bayesian regression over traditional frequentist methods has been identified by van Schoot and Depaoli (2014). This method is appropriate for parameter estimation with limited information available (Rossi and Allenby, 2003;Carlin and Louis, 2010;Sapsis, 2020) in (1) complex models, (2) when the researcher prefers the definition of probability, (3) when background knowledge can be incorporated into the analysis, and (4) only a small sample is available. Despite the number of individuals sampled in the current study (i.e., 24,000+), all data is aggregated at a country level for each of the 30 countries included, so our analysis is based on 30 observed cases.…”