Benchmarking the performance of Bayesian optimization across multiple experimental materials science domains

Abstract

Bayesian optimization (BO) has been leveraged for guiding autonomous and high-throughput experiments in materials science. However, few have evaluated the efficiency of BO across a broad range of experimental materials domains. In this work, we quantify the performance of BO with a collection of surrogate model and acquisition function pairs across five diverse experimental materials systems. By defining acceleration and enhancement metrics for materials optimization objectives, we find that surrogate models such as Gaussian Process (GP) with anisotropic kernels and Random Forest (RF) have comparable performance in BO, and both outperform the commonly used GP with isotropic kernels. GP with anisotropic kernels has demonstrated the most robustness, yet RF is a close alternative and warrants more consideration because it is free from distribution assumptions, has smaller time complexity, and requires less effort in initial hyperparameter selection. We also raise awareness about the benefits of using GP with anisotropic kernels in future materials optimization campaigns.

Benchmarking the Performance of Bayesian Optimization across Multiple Experimental Materials Science Domains

“…Across the datasets, the differences in distribution of normalized objective values can be observed in Figure 1(a); the differences in distribution of sampled data points in its respective materials design space can be seen in Figure 1(b). The five materials datasets in the current study are available in the following GitHub repository [38].…”

“…The five experimental datasets in the current study is available are the following GitHub repository [38]: https://github.com/PV-Lab/Benchmarking.…”

“…The code for pool-based active learning framework and visualization in the current study are available in the following GitHub repository [38]: https://github.com/PV-Lab/Benchmarking.…”

Machine Learning Optimization of Lignin Properties in Green Biorefineries

“…Recently, BO has become a popular tool in computational materials science, where it has been used both as a means of efficient global optimization [26][27][28] and to guide materials discovery [29]. BO is also increasingly being used for optimization in DOE applications [30][31][32][33][34]. Here, we highlight the power of BO's surrogate model aspect, which provides us with insight into lignin chemistry and facilitates easy multi-objective optimization for green chemistry applications.…”

Benchmarking Autonomous Scattering Experiments Illustrated on TAS

“…From our perspective, benchmarking and measuring performance is of the utmost importance at this stage for the establishment and progress of autonomous approaches in the field of materials analysis by scattering methods. As examples, similar efforts already developed a benchmark for oxygen evolution reaction catalyst discovery (Rohr et al, 2020) or evaluated the performance of Bayesian optimization (Frazier and Wang, 2016) across several materials science domains (Liang et al, 2021).…”