MOFormer: Self-Supervised Transformer Model for Metal–Organic Framework Property Prediction

Abstract

Metal−organic frameworks (MOFs) are materials with a high degree of porosity that can be used for many applications. However, the chemical space of MOFs is enormous due to the large variety of possible combinations of building blocks and topology. Discovering the optimal MOFs for specific applications requires an efficient and accurate search over countless potential candidates. Previous high-throughput screening methods using computational simulations like DFT can be time-consuming. Such methods also require the 3D atomic structures of MOFs, which adds one extra step when evaluating hypothetical MOFs. In this work, we propose a structure-agnostic deep learning method based on the Transformer model, named as MOFormer, for property predictions of MOFs. MOFormer takes a text string representation of MOF (MOFid) as input, thus circumventing the need of obtaining the 3D structure of a hypothetical MOF and accelerating the screening process. By comparing to other descriptors such as Stoichiometric-120 and revised autocorrelations, we demonstrate that MOFormer can achieve state-of-the-art structureagnostic prediction accuracy on all benchmarks. Furthermore, we introduce a self-supervised learning framework that pretrains the MOFormer via maximizing the cross-correlation between its structure-agnostic representations and structure-based representations of the crystal graph convolutional neural network (CGCNN) on >400k publicly available MOF data. Benchmarks show that pretraining improves the prediction accuracy of both models on various downstream prediction tasks. Furthermore, we revealed that MOFormer can be more data-efficient on quantum-chemical property prediction than structure-based CGCNN when training data is limited. Overall, MOFormer provides a novel perspective on efficient MOF property prediction using deep learning.

Predicting band gaps of MOFs on small data by deep transfer learning with data augmentation strategies

“…There have been many applications of ML in the materials eld [11][12][13] and satisfactory prediction performance has been achieved on the properties of MOFs. 7,14 However, advanced machine learning models (e.g. deep neural networks) require a large amount of data for training to make convincing predictions, which is difficult to meet for certain types of materials, such as porphyrin-based MOFs.…”

Denoise Pretraining on Nonequilibrium Molecules for Accurate and Transferable Neural Potentials

“…Also, GEM proposes to pretrain GNNs via predicting 3D positional information, including interatomic distances, bond lengths, and bond angles. On the other hand, contrastive learning, which aims at learning representations via contrasting positive instances against negative instances, has been widely implemented in GNNs for chemical sciences. − MolCLR applies random masking of atom and edge attributes to generate contrastive instances of molecular graphs. Recent works have also investigated multilevel subgraphs in contrastive training. , GraphMVP and 3D infomax incorporate 3D information into 2D graphs via contrasting molecules represented as 2D topological graphs and 3D geometric structures.…”

Denoise Pre-training on Non-equilibrium Molecules for Accurate and Transferable Neural Potentials

“…On the other hand, contrastive learning, which aims at learning representations via contrasting positive instances against negative instances, has been widely implemented in GNNs for chemical sciences. [54][55][56][57] MolCLR 58 applies random masking of atom and edge attributes to generate contrastive instances of molecular graphs. Recent works have also investigated multi-level subgraphs in contrastive training.…”