Computational design of stable and highly ion-conductive materials using multi-objective bayesian optimization: Case studies on diffusion of oxygen and lithium

Karasuyama, Masayuki; Kasugai, Hiroki; Tamura, Tomoyuki; Shitara, Kazuki

doi:10.1016/j.commatsci.2020.109927

Cited by 18 publications

(17 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We refer to this model as SAASBO. 3 For both models, we allow the model to infer noise levels rather than imposing a constant noise as an a-priori constraint or standard deviations supplied on a per-datapoint basis during the optimization 4 ; however, Ax is capable of handling these and other tasks including multi-objective optimization, risk-averse BO, batch-wise optimization, and custom surrogate models as well as a variety of other non-algorithm related features 5 . Because the computational overhead of SAASBO scales cubically with the number of datapoints, it is typically limited to several hundred adaptive design iterations which is appropriate for our expensive hyperparameter optimization use-case.…”

Section: Ax Bayesian Optimizationmentioning

confidence: 99%

“…3 23 hyperparameters might not seem large at first; but for a design budget of only 100 iterations, finding optimal hyperparameter is a difficult problem. 4 Repeat measurements may also be supplied directly rather than imposing the assumption of a particular noise distribution, e.g. Gaussian.…”

Section: Ax Bayesian Optimizationmentioning

confidence: 99%

“…Bayesian optimization (BO) is an iterative sequential learning 1 algorithm that simultaneously improves model accuracy through exploration of high-uncertainty regions and exploitation of highperforming parameter combinations. It is bestsuited for expensive-to-evaluate models with a limited budget of design iterations, and has seen increasing usage in materials informatics [1][2][3][4][5][6][7].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

High-dimensional Bayesian Optimization of Hyperparameters for an Attention-based Network to Predict Materials Property: a Case Study on CrabNet using Ax and SAASBO

Baird¹,

Liu²,

Sparks³

2022

Preprint

View full text Add to dashboard Cite

Expensive-to-train deep learning models can benefit from an optimization of the hyperparameters that determine the model architecture. We optimize 23 hyperparameters of a materials informatics model, Compositionally-Restricted Attention-Based Network (CrabNet), over 100 adaptive design iterations using two models within the Adaptive Experimentation (Ax) Platform. This includes a recently developed Bayesian optimization (BO) algorithm, sparse axis-aligned subspaces Bayesian optimization (SAASBO), which has shown exciting performance on high-dimensional optimization tasks. Using SAASBO to optimize CrabNet hyperparameters, we demonstrate a new state-of-the-art on the experimental band gap regression task within the materials informatics benchmarking platform, Matbench (∼4.5 % decrease in mean absolute error (MAE) relative to incumbent). Characteristics of the adaptive design scheme as well as feature importances are described for each of the Ax models. SAASBO has great potential to both improve existing surrogate models, as shown in this work, and in future work, to efficiently discover new, high-performing materials in high-dimensional materials science search spaces.

show abstract

Section: Ax Bayesian Optimizationmentioning

confidence: 99%

Section: Ax Bayesian Optimizationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

High-dimensional Bayesian Optimization of Hyperparameters for an Attention-based Network to Predict Materials Property: a Case Study on CrabNet using Ax and SAASBO

Baird¹,

Liu²,

Sparks³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The datasets are synthesized by physical-simulations, and X is a set defined by finite candidate structures. We used two datasets called Bi 2 O 3 and LLTO from [18]. The number of candidate points in Bi permuted.…”

Section: Applications To Materials Datamentioning

confidence: 99%

Sequential- and Parallel- Constrained Max-value Entropy Search via Information Lower Bound

Takeno¹,

Tamura

Shitara

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Recently, several Bayesian optimization (BO) methods have been extended to the expensive black-box optimization problem with unknown constraints, which is an important problem that appears frequently in practice. We focus on an information-theoretic approach called Max-value Entropy Search (MES) whose superior performance has been repeatedly shown in BO literature. Since existing MES-based constrained BO is restricted to only one constraint, we first extend it to multiple constraints, but we found that this approach can cause negative approximate values for the mutual information, which can result in unreasonable decisions. In this paper, we employ a different approximation strategy that is based on a lower bound of the mutual information, and propose a novel constrained BO method called Constrained Maxvalue Entropy Search via Information lower BOund (CMES-IBO). Our approximate mutual information derived from the lower bound has a simple closed-form that is guaranteed to be nonnegative, and we show that irrational behavior caused by the negative value can be avoided. Furthermore, by using conditional mutual information, we extend our methods to the parallel setting in which multiple queries can be issued simultaneously. Finally, we demonstrate the effectiveness of our proposed methods by benchmark functions and real-world applications to materials science.

show abstract

“…In this paper, this inverse design approach is called “many-solution-inverse-design”. Some recent computational material design studies used MO BO in many-solution-inverse-design to accelerate the finding of the Pareto optimal solutions ( Janet et al., 2020 ; Karasuyama et al., 2020 ; Solomou et al., 2018 ; Talapatra et al., 2018 ; Wang et al., 2020b ). However, as schematically shown in Figure 2 A, finding many Pareto optimal solutions requires an excessive number of experiments, which is difficult to execute with time-consuming real-world experiments.…”

Section: Introductionmentioning

confidence: 99%

Bayesian optimization for goal-oriented multi-objective inverse material design

Hanaoka

2021

iScience

View full text Add to dashboard Cite

Bayesian optimization (BO) can accelerate material design requiring timeconsuming experiments. However, although most material designs require tuning of multiple properties, the efficiency of multi-objective (MO) BO in timeconsuming experimental material design remains unclear, due to the complexity of handling multiple objectives. This study introduces MO BO method that efficiently achieves predefined goals and shows that by focusing on achieving the goals, BO can efficiently accelerate realistic MO design problems with small efforts. Benchmarks showed that the proposed BO method dramatically reduced the number of experiments needed to achieve goals relative to a baseline method. Virtual MO inverse design experiments with realistic material design problems were also performed, during which the proposed method could achieve goals within only around ten experiments in average and showed over 1000-fold acceleration relative to the random sampling for the most difficult case. The introduction of goal-oriented BO will precede real-world application of BO.

show abstract

Computational design of stable and highly ion-conductive materials using multi-objective bayesian optimization: Case studies on diffusion of oxygen and lithium

Cited by 18 publications

References 55 publications

High-dimensional Bayesian Optimization of Hyperparameters for an Attention-based Network to Predict Materials Property: a Case Study on CrabNet using Ax and SAASBO

High-dimensional Bayesian Optimization of Hyperparameters for an Attention-based Network to Predict Materials Property: a Case Study on CrabNet using Ax and SAASBO

Sequential- and Parallel- Constrained Max-value Entropy Search via Information Lower Bound

Bayesian optimization for goal-oriented multi-objective inverse material design

Contact Info

Product

Resources

About