Metamodelling or surrogate modelling techniques are frequently used across the engineering disciplines in conjunction with expensive simulation models or physical experiments. With the proliferation of metamodeling techniques developed to provide enhanced performance for specific problems, and the wide availability of a diverse choice of tools in engineering software packages, the engineering task of selecting a robust metamodeling technique for practical problems is still a challenge. This research introduces a framework for describing the typology of engineering problems, in terms of dimensionality and complexity, and the modelling conditions, reflecting the noisiness of the signals and the affordability of sample sizes, and on this basis presents a systematic evaluation of the performance of frequently used metamodeling techniques. A set of metamodeling techniques, selected based on their reported use for engineering problems (i.e. Polynomial, Radial Basis Function, and Kriging), were systematically evaluated in terms of accuracy and robustness against a carefully assembled set of 18 test functions covering different types of problems, sampling conditions and noise conditions. A set of four real-world engineering case studies covering both computer simulation and physical experiments were also analysed as validation tests for the proposed guidelines. The main conclusions drawn from the study are that Kriging model with Matérn 5/2 correlation function performs consistently well across different problem types with smooth (i.e. not noisy) data, while Kriging model with Matérn 3/2 correlation function provides robust performance under noisy conditions, except for the very high noise conditions, where the Kriging model with nugget appears to provide better models. These results provide engineering practitioners with a guide for the choice of a metamodeling technique for problem types and modelling conditions represented in the study, whereas the evaluation framework and benchmarking problems set will be useful for researchers conducting similar studies. Structural and Multidisciplinary Optimization (2020) 61:159-186 Performance evaluation of metamodelling methods for engineering problems: towards a practitioner guide 161

The work presented in this paper is motivated by a complex multivariate engineering problem associated with engine mapping experiments, which require efficient design of experiments (DoE) strategies to minimise expensive testing. The paper describes the development and evaluation of a Permutation Genetic Algorithm (PermGA) to enable an exploration-based sequential DoE strategy for complex reallife engineering problems. A known PermGA was implemented to generate uniform OLH DoEs, and substantially extended to support generation of model building-model validation (MB-MV) sequences, by generating optimal infill sets of test points as OLH DoEs that preserve good spacefilling and projection properties for the merged MB + MV test plan. The algorithm was further extended to address issues with non-orthogonal design spaces, which is a common problem in engineering applications. The effectiveness of the PermGA algorithm for the MB-MV OLH DoE sequence was evaluated through a theoretical benchmark problem based on the Six-Hump-Camel-Back function, as well as the Gasoline Direct Injection engine steady-state engine mapping problem that motivated this research. The case studies show that the algorithm is effective in delivering quasi-orthogonal spacefilling DoEs with good properties even after several MB-MV iterations, while the improvement in model adequacy and accuracy can be monitored by the engineering analyst. The Communicated by

No abstract

This paper presents the development and implementation of a customised Permutation Genetic Algorithm (PermGA) for a sequential Design of Experiment (DoE) framework based on space filling Optimal Latin Hypercube (OLH) designs. The work is motivated by multivariate engineering problems such as engine mapping experiments, which require efficient DoE strategies to minimise expensive testing. The DoE strategy is based on a flexible Model BuildingModel Validation (MB-MV) sequence based on space filling OLH DoEs, which preserves the space filling and projection properties of the DoEs through the iterations. A PermGA algorithm was developed to generate MB OLHs, subsequently adapted for generation of infill MV test points as OLH DoEs, preserving good space filling and projection properties for the merged MB + MV test plan. The algorithm was further modified to address issues with non-orthogonal design spaces. A case study addressing the steady state engine mapping of a Gasoline Direct Injection was used to illustrate and validate the practical application of MB-MV sequence based on the developed PermGA algorithm.

No abstract