Combining multi-fidelity modelling and asynchronous batch Bayesian Optimization

Bayesian optimisation for efficient material discovery: a mini review

“…J. P. Folch et al found that MOGP outperforms independent GP. 56 Novel acquisition functions are also developed to conduct MFBO. For instance, K. Kandasamy et al proposed multi-fidelity UCB (MF-UCB) and introduced the concept of fidelity threshold such that the low fidelity data are used for exploration while the high fidelity data are used to exploit promising regions.…”

“…However, this method could constrain transfer learning between fidelities. 56 One approach is through the multi-output Gaussian process (MOGP) where the result from each fidelity will serve as one output in the MOGP model. J. P. Folch et al found that MOGP outperforms independent GP.…”

“…57 J. P. Folch et al then furthered MF-UCB with batch optimisation and applied this combination to design the cathode material for batteries. 56 In a similar vein, Z. Z. Foumani et al proposed a cost-aware MF acquisition function that performs exploration and exploitation using low and high fidelity data, respectively.…”

Outlook: How I Learned to Love Machine Learning (A Personal Perspective on Machine Learning in Process Systems Engineering)

“…Analogous to the story on how capabilities of optimization were extended by the PSE community to tackle complex problems, I would like to highlight some of the work that the PSE community has been doing in extending the capabilities of ML. The PSE community has developed powerful global optimization algorithms and software that can handle formulations that have embedded neural network and GP models. , The community has also recently developed BO architectures that combine data-driven and physics models (and models of different levels of resolution) to guide experimental design. ,, Moreover, the community has extensively explored the use of ML models as surrogates of complex models. ,, Along these lines, I would like to highlight the development of , which is a software package for optimization modeling that automates the conversion of ML models (e.g., nonlinear neural networks) into tractable, mixed-integer linear representations . I believe that this work can be highly impactful, as it can help standardize modeling environments (e.g., every unit operation in a chemical process is a neural network).…”

“…Specifically, PSE can play a key role in finding suitable data representations for molecules, chemical reactions, dynamical systems, flowsheets, and expert logic (and connections between them); such representations can then be fed to ML tools to conduct diverse tasks. For instance, recent work by the PSE community has explored data representations and ML models to predict molecular properties. ,, Recent work by the PSE community has also developed data representations of flowsheets as graphs and text-strings (analogous to SMILES strings) and has used these to train ML models that can automatically synthesize flowsheets. − ML tools such as physics-informed neural networks and physics-constrained neural networks also provide hybrid modeling capabilities that allow PSE researchers to fuse data-driven and physical models in new ways. − The PSE community has also developed new control, optimization, scheduling, and experimental design formulations that make use of ML techniques. ,− All this work is a clear example of how PSE leverages tools of ML to come up with innovative abstractions that facilitate discovery and decision-making. It is important to highlight that the PSE community was an early adopter of ML tools such as neural networks (going back to the 1980s and 1990s), but this early adoption was not as widespread.…”