Identifying Domains of Applicability of Machine Learning Models for Materials Science

Sutton, Christopher; Boley, Mario; Ghiringhelli, Luca M.; Rupp, Matthias; Vreeken, Jilles; Scheffler, Matthias

doi:10.26434/chemrxiv.9778670.v2

Cited by 7 publications

(7 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…GPR is a kernel method in which structure-property relationships are inferred from similarities between molecules. For measuring such similarities, in GPR as in any ML approach in chemistry, it is important to choose a unique and information-conserving way of translating molecular structures into ML-accessible format (molecular representations / descriptors) 34,49,55,[90][91][92][93][94][95][96][97] .…”

Section: Introductionmentioning

confidence: 99%

Exchange Spin Coupling from Gaussian Process Regression

et al. 2020

View full text Add to dashboard Cite

Heisenberg exchange spin coupling between metal centers is essential for describing and understanding the electronic structure of many molecular catalysts, metalloenzymes, and molecular magnets for potential application in information technology. We explore the machine-learnability of exchange spin coupling, which has not been studied yet. We employ Gaussian process regression since it can potentially deal with small training sets (as likely associated with the rather complex molecular structures required for exploring spin coupling) and since it provides uncertainty estimates ("error bars") along with predicted values. We compare a range of descriptors and kernels for 257 small dicopper complexes and find that a simple descriptor based on chemical intuition, consisting only of copper-bridge angles and copper-copper distances, clearly outperforms several more sophisticated descriptors when it comes to extrapolating towards larger experimentally relevant complexes. Exchange spin coupling is similarly easy to learn as the polarizability, while learning dipole moments is much harder. The strength of the sophisticated descriptors lies in their ability to linearize structure-property relationships, to the point that a simple linear ridge regression performs just as well as the kernel-based machine-learning model for our small dicopper data set. The superior extrapolation performance of the simple descriptor is unique to exchange spin coupling, reinforcing the crucial role of choosing a suitable descriptor, and highlighting the interesting question of the role of chemical intuition vs. systematic or automated selection of features for machine learning in chemistry and material science. File list (4) download file view on ChemRxiv supp.pdf (2.07 MiB) download file view on ChemRxiv ms.pdf (5.17 MiB) download file view on ChemRxiv Cu2-dataset.zip (22.06 MiB) download file view on ChemRxiv py_regression.zip (17.79 MiB)

show abstract

Section: Introductionmentioning

confidence: 99%

Exchange Spin Coupling from Gaussian Process Regression

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Given that there are no corresponding features in the mergedpharmacophore, it is acceptable to disregard potentially missing features during prediction since no information about their contribution to activity in that location is known. Therefore, including these features at inference would only increase noise and weaken the model's con dence in the prediction (23).…”

Section: Quantitative Pharmacophore Algorithmmentioning

confidence: 99%

QPhAR – Quantitative Pharmacophore Activity Relationship: Method and Validation

Kohlbacher

Langer

Seidel

2021

Preprint

View full text Add to dashboard Cite

QSAR methods are widely applied in the drug discovery process, both in the hit‑to‑lead and lead optimization phase, as well as in the drug-approval process. Most QSAR algorithms are limited to using molecules as input and disregard pharmacophores or pharmacophoric features entirely. However, due to the high level of abstraction, pharmacophore representations provide some advantageous properties for building quantitative SAR models. The abstract depiction of molecular interactions avoids a bias towards overrepresented functional groups in small datasets. Furthermore, a well‑crafted quantitative pharmacophore model can generalise to underrepresented or even missing molecular features in the training set by using pharmacophoric interaction patterns only. This paper presents a novel method to construct quantitative pharmacophore models and demonstrates its applicability and robustness on more than 250 diverse datasets. 5‑fold cross-validation on these datasets with default settings yielded an average RMSE of 0.62, with an average standard deviation of 0.18. Additional cross-validation studies on datasets with 15-20 training samples showed that robust quantitative pharmacophore models could be obtained. These low requirements for dataset sizes renders quantitative pharmacophores a viable go-to method for medicinal chemists, especially in the lead-optimisation stage of drug discovery projects.

show abstract

“…It is difficult to generate data sets that properly reflect the search space, and as such data sets may be biased and result in an overly optimistic assessment of model efficacy. 32,33 A recent analysis 1 of several supervised learning approaches to materials stability [2][3][4][5][6][7] showed that, despite being able to learn DFT formation energies with reasonable accuracy, the methods struggled to reproduce DFT decomposition energies. The difference was attributed to the observation that DFT-computed energies benefit from a systematic cancellation of errors not present in ML, that helps DFT better distinguish relative formation energies.…”

Section: Introductionmentioning

confidence: 99%

A Combined DFT/Machine Learning Framework for Materials Discovery: Application to Spinels and Assessment of Search Completeness and Efficiency

Ertekin¹,

Schiller²

2020

Preprint

View full text Add to dashboard Cite

It is challenging to evaluate machine learning approaches developed for accelerating materials search and discovery in a realistic way. Machine learning approaches to materials stability prediction are typically assessed by their ability to reproduce results from direct physical modeling, whereas ideally both machine learning and direct physical modeling should be assessed by their ability to reproduce reality. Additionally, traditional evaluation metrics do not directly reflect the experience of an experimental search for unknown compounds in a large candidate phase space, and often result in overly optimistic assessments. Here, we (i) present a framework that combines density functional theory and traditional supervised machine learning methods (ML/DFT), and (ii) introduce the concepts of search completeness – the fraction of discoverable compounds found relative to the fraction of search space explored – and search efficiency – the rate of discovery relative to the fraction of search space explored – to evaluate it. The ML/DFT framework is an iterative approach to predict stable chemistries of a fixed crystal structure (here, spinels) that uses DFT to generate a training set of unstable compounds. The training set of stable compounds is given by experimentally known spinels. The method is carried out using random forest, LASSO, and ridge regression to predict as-of-yet undiscovered spinel chemistries. TreeSHAP analysis is used to determine features that most contribute to stability/instability classification. While no single feature dominates, several emerge that align with chemical intuition. To estimate the efficacy of ML/DFT compared to pure DFT, we introduce a Bayesian description of DFT distribution of energies for stable and unstable spinels. The Bayesian model enables quantifying the search completeness and search efficiency of DFT, which is then compared to that of ML/DFT. ML/DFT achieves search completeness and efficiency on par with pure DFT, despite requiring fewer DFT simulations (∼300 vs. 14,200). More importantly, by quantitatively assessing ML approaches in ways that better reflect how they would be used in materials discovery experiments, we obtain key insights into the challenges that need to be overcome by such methods: that the small number of stable compounds to be found in a search space orders of magnitude larger places stringent demands on model accuracy to achieve good search efficiency. Finally, we report the top candidates of our spinel search, which may be of interest for synthesis experiments<br>

show abstract

Identifying Domains of Applicability of Machine Learning Models for Materials Science

Cited by 7 publications

References 32 publications

Exchange Spin Coupling from Gaussian Process Regression

Exchange Spin Coupling from Gaussian Process Regression

QPhAR – Quantitative Pharmacophore Activity Relationship: Method and Validation

A Combined DFT/Machine Learning Framework for Materials Discovery: Application to Spinels and Assessment of Search Completeness and Efficiency

Contact Info

Product

Resources

About