Sensitivity analysis of expensive black-box systems using metamodeling

Abstract

Simulations are becoming ever more common as a tool for designing complex products. Sensitivity analysis techniques can be applied to these simulations to gain insight, or to reduce the complexity of the problem at hand. However, these simulators are often expensive to evaluate and sensitivity analysis typically requires a large amount of evaluations. Metamodeling has been successfully applied in the past to reduce the amount of required evaluations for design tasks such as optimization and design space exploration. In this paper, we propose a novel sensitivity analysis algorithm for variance and derivative based indices using sequential sampling and metamodeling. Several stopping criteria are proposed and investigated to keep the total number of evaluations minimal. The results show that both variance and derivative based techniques can be accurately computed with a minimal amount of evaluations using fast metamodels and FLOLA-Voronoi or density sequential sampling algorithms.

IEEE 802.11ah Restricted Access Window Surrogate Model for Real-Time Station Grouping

“…This process continues until the stop conditions are met. In our experiments, the SUMO toolbox is configured to use the latin hypercube sampling method [19] to generate 100 initial sample data points, Kriging interpolation is used to train the model [20], FLOLA-Voronoi sampling for generating the next sample points [21], and the 10-fold cross-validation with a root-relative square error (RRSE) measure to evaluate the model accuracy [22]. The training stops once the crossvalidation score lower than or equal to 0.10 (2 digits of precision) occurs 10 times in succession, or the number of training data points exceeds 2500.…”

Surrogate Modelling with Sequential Design for Expensive Simulation Applications

“…The surrogate models can be used directly for evaluation-based sensitivity analysis methods such as Sobol indices [4], Interaction indices [5] or gradient-based methods. For some kernel-based modelling methods, analytical computation of sensitivity measures is possibly resulting in faster and more reliable estimation schemes, even before global accuracy is achieved [6].…”

“…When 300 samples were evaluated, the process was terminated and the analytical approach presented in Ref. [6] was used to compute the first-order Sobol indices, as well as the total Sobol indices (first-order indices augmented with all indices of higher order interactions containing this parameter) [4]. Both indices are plotted in Figure 9.…”

Batch Bayesian Active Learning For Feasible Region Identification by Local Penalization

“…An appropriate metamodel to approximate the continuous function f with limited observations is Kriging (Forrester et al 2008, Van Steenkiste et al 2016, Rojas-Gonzalez et al 2019. Kriging is also known as a Gaussian Process (GP) (Rasmussen 2003): p( f ) = GP(µ, k), where µ : X → R (assumed zero in this research) is the mean function, and k : X × X → R is the kernel.…”