Uni-Mol: A Universal 3D Molecular Representation Learning Framework

Abstract

Molecular representation learning (MRL) has gained tremendous attention due to its critical role in learning from limited supervised data for applications like drug design. In most MRL methods, molecules are treated as 1D sequential tokens or 2D topology graphs, limiting their ability to incorporate 3D information for downstream tasks and, in particular, making it almost impossible for 3D geometry prediction or generation. Herein, we propose Uni-Mol, a universal MRL framework that significantly enlarges the representation ability and application scope of MRL schemes. Uni-Mol is composed of two models with the same SE(3)-equivariant transformer architecture: a molecular pretraining model trained by 209M molecular conformations; a pocket pretraining model trained by 3M candidate protein pocket data. The two models are used independently for separate tasks, and are combined when used in protein-ligand binding tasks. By properly incorporating 3D information, Uni-Mol outperforms SOTA in 14/15 molecular property prediction tasks. Moreover, Uni-Mol achieves superior performance in 3D spatial tasks, including protein-ligand binding pose prediction, molecular conformation generation, etc. Finally, we show that Uni-Mol can be successfully applied to the tasks with few-shot data like pocket druggability prediction. The model and data will be made publicly available at \url{https://github.com/dptech-corp/Uni-Mol}

Do Deep Learning Models Really Outperform Traditional Approaches in Molecular Docking?

“…Recently, deep learning has been widely used in drug design applications, such as molecular property prediction [5,6,7,8] and protein structure predition [9]. Several recent works [10,11,12] also try to apply deep learning to molecular docking.…”

3D Graph Contrastive Learning for Molecular Property Prediction

“…We evaluate the 3DGCL performance with standard supervised baselines and self-supervised models. All compared methods use more than one dataset of six benchmark datasets (ESOL, Freesolv, QM7, QM8, BBBP, and BACE) and conduct experiments under the same condition (Zhou et al, 2022). The same condition denotes scaffold-splitting (with considering chirality) the train/validation/test data to an 8:1:1 ratio and running tests independently three times with three random seeds.…”

“…For a fair comparison with the state-of-the-art models, we run the model with random seeds three times and average the performance and standard deviation in the same way in previous works. In the same way (Wang et al, 2022;Rong et al, 2020;Zhou et al, 2022), we evaluated QM7 for 1 target task and QM8 for the average of 12 target tasks.…”

“…MEMO (Zhu et al, 2022) is a multiple-view contrastive learning method using various molecular representations (2D graph, 3D graph, Fingerprint, and SMILES). Uni-Mol (Zhou et al, 2022) is a self-supervised model that uses 3D information and consists of a pre-training scheme that predicts masked atoms or denoises 3D positions after adding noise to the molecular coordinates.…”

DR-Label: Improving GNN Models for Catalysis Systems by Label Deconstruction and Reconstruction

“…Developing an effective machine learning-based model to predict the equilibrium state and property is of high interest far beyond the catalyst adsorption energy prediction area. Similar models and techniques can also be transferred to other pivotal topics such as drug discovery Zhou et al, 2022), protein structure forecast (Jing et al, 2020;Jumper et al, 2021), and even in engineering design (Pfaff et al, 2020). Graph neural networks (GNNs) currently play a predominant role in this area.…”