Efficient space-filling and non-collapsing sequential design strategies for simulation-based modeling

Abstract

Simulated computer experiments have become a viable cost-effective alternative for controlled real-life experiments. However, the simulation of complex systems with multiple input and output parameters can be a very timeconsuming process. Many of these high-fidelity simulators need minutes, hours or even days to perform one simulation. The goal of global surrogate modeling is to create an approximation model that mimics the original simulator, based on a limited number of expensive simulations, but can be evaluated much faster. The set of simulations performed to create this model is called the experimental design. Traditionally, one-shot designs such as the Latin hypercube and factorial design are used, and all simulations are performed before the first model is built. In order to reduce the number of simulations needed to achieve the desired accuracy, sequential design methods can be employed. These methods generate the samples for the experimental design one by one, without knowing the total number of samples in advance. In this paper, the authors perform an extensive study of new and state-of-the-art space-filling sequential design methods. It is shown that the new sequential methods proposed in this paper produce results comparable to the best one-shot experimental designs available right now.

Adaptive classification under computational budget constraints using sequential data gathering

“…This section describes the concept of sequential design, the default experimental design method of the toolbox, and explains how it differs from one-shot experimental design [14,15,2] methodology as used traditionally in the design and analysis of computer experiments. This methodology can be applied when the output of the computer experiments is continuous (regression, as usually encountered in surrogate modeling) and for discrete outputs (classification) as described in this article.…”

“…Distributed computing support for evaluations of data points is also available, as well as multi-threading to support the usage of multi-core architectures for regression modeling and classification. Many different plugins are available for each of the different sub-problems: model types (rational functions, Kriging [5], splines, Support Vector Machines (SVM) [6,7,8], Artificial Neural Networks (ANN), Extreme Learning Machines (ELM) [9], Least Squares-SVM (LS-SVM) [10], Random Forests [11]), hyperparameter optimization algorithms (Particle Swarm Optimization [12], Efficient Global Optimization [13], simulated annealing, Genetic Algorithm), sample selection (random, error based, density based [14,15], hybrid [16]), Design of Experiments (Latin hypercube [17,18], Box-Bhenken), and sample evaluation methods (local, on a cluster or grid). The behavior of each software component is configurable through a central XML file and components can easily be added, removed or replaced by custom implementation.…”

Sensitivity analysis of expensive black-box systems using metamodeling

“…Another sequential sampling technique is the density based design, introduced in (Crombecq et al 2011). In this method, points are generated taking into account the maximin distance and the projected distance.…”

Adaptive modeling and sampling methodologies for Internet of Things applications

“…Hyperparameter optimization algorithms include Particle Swarm Optimization [15], Efficient Global Optimization [16] (commonly referred to as Bayesian optimization), simulated annealing, and Genetic Algorithms. Model selection can be done with cross-validation, but also AIC, a Leaveout set and LRM [17] are available, as well as sample selection approaches such as random, error based, density based [18], [19] or hybrid methods [20]. Popular Design of Experiments methodologies for computer experiments such as Latin hypercube [21], [22] or Box-Bhenken are included as well.…”