Graph Networks as a Universal Machine Learning Framework for Molecules and Crystals

Abstract

Graph networks are a new machine learning (ML) paradigm that supports both relational reasoning and combinatorial generalization. Here, we develop universal Ma-tErials Graph Network (MEGNet) models for accurate property prediction in both molecules and crystals. We demonstrate that the MEGNet models outperform prior ML models such as the SchNet in 11 out of 13 properties of the QM9 molecule data set. Similarly, we show that MEGNet models trained on ∼ 60, 000 crystals in the Materials Project substantially outperform prior ML models in the prediction of the formation energies, band gaps and elastic moduli of crystals, achieving better than DFT accuracy over a much larger data set. We present two new strategies to address data limitations common in materials science and chemistry. First, we demonstrate a physically-intuitive approach to unify four separate molecular MEGNet models for the internal energy at 0 K and room temperature, enthalpy and Gibbs free energy into a single free energy MEGNet model by incorporating the temperature, pressure and entropy as global state inputs. Second, we show that the learned element embeddings in MEGNet models encode periodic chemical trends and can be transfer-learned from arXiv:1812.05055v2 [cond-mat.mtrl-sci] a property model trained on a larger data set (formation energies) to improve property models with smaller amounts of data (band gaps and elastic moduli).

A Critical Review of Machine Learning of Energy Materials

“…Finally, graph‐based featurization has gained substantial interest in recent years. Graphs, which are natural representations for atoms (nodes) and the bonds between them (edges), have been used for molecules for many decades and have recently been applied to ML in crystals, achieving state‐of‐the‐art performance in predicting the formation energies, bandgaps, as well as metal/insulator classification …”

“…While PCA works on linear projection, the manifold learning is able to capture nonlinear relationships. For example, the t‐distributed stochastic neighbor embedding (t‐SNE) method learns low‐dimensional representations such that the local distance between data points is roughly preserved and has been applied in visualizing the elemental embedding vector trained from materials property prediction models, structural similarity of perovskites, word embeddings in text mining, electronic fingerprints, etc.…”

“…Owing to the explosion in interest in materials ML, there has been a proliferation of open‐source software tools to facilitate featurization of crystals/molecules. A noncomprehensive list includes RDKit, Dscribe, Matminer, Materials Agnostic Platform for Informatics and Exploration (Magpie), MatErials Graph Network (MEGNet), etc. Interested readers may wish to explore these tools for their ML projects.…”

“…Modified graph models can also update bond information using information from atoms that form the bond . More recently, graph networks further generalize the GCNN by introducing global attributes in addition to atom and bond attributes, and allowing information flow among all three levels of quantities . Graph‐based deep learning models have shown remarkable performance in molecular and crystal property predictions compared to other ML models …”

“…Besides general purpose ML tools such as scikit‐learn, tensorflow, and Pytorch, there has been an explosion of customized open‐source ML software libraries for materials science. A nonexhaustive list includes AutoMatminer, PROPhet for general materials ML; amp, ænet, and ANI for developing neural network potentials; CGCNN, MEGNet, and SchnetPack are graph‐based deep learning model packages for accurate crystal and/or molecule property modeling.…”