A survey on high-dimensional Gaussian process modeling with application to Bayesian optimization

Binois, Mickaël; Wycoff, Nathan

doi:10.48550/arxiv.2111.05040

Cited by 6 publications

(7 citation statements)

References 77 publications

(123 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Regarding the optimization procedure, a natural evolution is the implementation of a multi-objective optimization which considers at the same time both mass and deflection at a given point, for example. This can be done in a Bayesian setting, accounting for high dimensional input parameter space, 68,69 but also other approaches should be considered, such as genetic algorithms enhanced by active subspaces. 70,71 by SMACT Competence Center, and partially funded by European Union Funding for Research and Innovation -Horizon 2020 Program -in the framework of European Research Council Executive Agency: H2020 ERC CoG 2015 AROMA-CFD project 681447 "Advanced Reduced Order Methods with Applications in Computational Fluid Dynamics" P.I.…”

Section: Discussionmentioning

confidence: 99%

A multifidelity approach coupling parameter space reduction and nonintrusive POD with application to structural optimization of passenger ship hulls

Tezzele

Fabris

Sidari

et al. 2022

Numerical Meth Engineering

View full text Add to dashboard Cite

Nowadays, the shipbuilding industry is facing a radical change toward solutions with a smaller environmental impact. This can be achieved with low emissions engines, optimized shape designs with lower wave resistance and noise generation, and by reducing the metal raw materials used during the manufacturing. This work focuses on the last aspect by presenting a complete structural optimization pipeline for modern passenger ship hulls which exploits advanced model order reduction techniques to reduce the dimensionality of both input parameters and outputs of interest. We introduce a novel approach which incorporates parameter space reduction through active subspaces into the proper orthogonal decomposition with interpolation method. This is done in a multi‐fidelity setting. We test the whole framework on a simplified model of a midship section and on the full model of a passenger ship, controlled by 20 and 16 parameters, respectively. We present a comprehensive error analysis and show the capabilities and usefulness of the methods especially during the preliminary design phase, finding new unconsidered designs while handling high dimensional parameterizations.

show abstract

Section: Discussionmentioning

confidence: 99%

A multifidelity approach coupling parameter space reduction and nonintrusive POD with application to structural optimization of passenger ship hulls

Tezzele

Fabris

Sidari

et al. 2022

Numerical Meth Engineering

View full text Add to dashboard Cite

show abstract

“…The limiting factor for scalability is the training of the GP emulator, which scales cubically with the number of training points as a result of having to perform matrix inversion (or pseudo-inversion) of a covariance matrix. A growing number of techniques exist that improve the scalability of GPs, for example, by identifying and removing variables with little or no impact on the output, or by defining new variables based on combinations of the original ones (e.g., Binois & Wycoff, 2021). Alternatively, other machine learning models such as deep neural networks can be used that scale better with input dimensions (e.g., Lan et al, 2021).…”

Section: Summary and Discussionmentioning

confidence: 99%

An Efficient Bayesian Approach to Learning Droplet Collision Kernels: Proof of Concept Using “Cloudy,” a New n‐Moment Bulk Microphysics Scheme

Bieli

Dunbar

Jong

et al. 2022

J Adv Model Earth Syst

View full text Add to dashboard Cite

Historically, microphysics schemes were tuned to data in an ad-hoc way, resulting in parameter values that are not repeatable or explainable • Bayesian inference puts uncertainty quantification and parameter learning on solid mathematical grounds, but is computationally expensive • We present a proof-of-concept of computationally efficient Bayesian learning applied to a new bulk microphysics scheme called "Cloudy"

show abstract

“…We point the reader to [BW21] for a comprehensive overview of the state-of-the-art in high-dimensional BO.…”

Section: Related Workmentioning

confidence: 99%

“…The maximization task of the acquisition function is also hampered by high dimensionality. As such, BO is often taken only for small-scale problems (typically less than 20 search variables), and it remains an open challenge to scale it up for high-dimensional problems [BW21].…”

Section: Introductionmentioning

confidence: 99%

High Dimensional Bayesian Optimization with Kernel Principal Component Analysis

Antonov¹,

Raponi²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Bayesian Optimization (BO) is a surrogate-based global optimization strategy that relies on a Gaussian Process regression (GPR) model to approximate the objective function and an acquisition function to suggest candidate points. It is well-known that BO does not scale well for high-dimensional problems because the GPR model requires substantially more data points to achieve sufficient accuracy and acquisition optimization becomes computationally expensive in high dimensions. Several recent works aim at addressing these issues, e.g., methods that implement online variable selection or conduct the search on a lower-dimensional sub-manifold of the original search space. Advancing our previous work of PCA-BO that learns a linear sub-manifold, this paper proposes a novel kernel PCA-assisted BO (KPCA-BO) algorithm, which embeds a non-linear sub-manifold in the search space and performs BO on this sub-manifold. Intuitively, constructing the GPR model on a lower-dimensional sub-manifold helps improve the modeling accuracy without requiring much more data from the objective function. Also, our approach defines the acquisition function on the lower-dimensional sub-manifold, making the acquisition optimization more manageable. We compare the performance of KPCA-BO to the vanilla BO and PCA-BO on the multi-modal problems of the COCO/BBOB benchmark suite. Empirical results show that KPCA-BO outperforms BO in terms of convergence speed on most test problems, and this benefit becomes more significant when the dimensionality increases. For the 60D functions, KPCA-BO surpasses PCA-BO in many test cases. Moreover, it efficiently reduces the CPU time required to train the GPR model and optimize the acquisition function compared to the vanilla BO.

show abstract

A survey on high-dimensional Gaussian process modeling with application to Bayesian optimization

Cited by 6 publications

References 77 publications

A multifidelity approach coupling parameter space reduction and nonintrusive POD with application to structural optimization of passenger ship hulls

A multifidelity approach coupling parameter space reduction and nonintrusive POD with application to structural optimization of passenger ship hulls

An Efficient Bayesian Approach to Learning Droplet Collision Kernels: Proof of Concept Using “Cloudy,” a New n‐Moment Bulk Microphysics Scheme

High Dimensional Bayesian Optimization with Kernel Principal Component Analysis

Contact Info

Product

Resources

About