GEM-2: Next Generation Molecular Property Prediction Network by Modeling Full-range Many-body Interactions

Liu, Lihang; He, Donglong; Fang, Xiaomin; Zhang, Shanzhuo; Wang, Fan; He, Jingzhou; Wu, Hua

doi:10.21203/rs.3.rs-2151362/v1

Cited by 6 publications

(7 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Baselines. Because the label of test datasets is officially hidden, we compared the results of validation with the top-tier method on the OGB leaderboard 2 , which includes GRPE [34], TokenGT [20], EGT [18] , GPS [40], GEM-2 [26], Vis-Net [52] and Transformer-M [29]. In addition, we also compared GPS++ [30], the winner solution at OGB LSC @ NeruIPS 2022 Challenge.…”

Section: Pre-training On Large-scale Datasetmentioning

confidence: 99%

Automated 3D Pre-Training for Molecular Property Prediction

Wang¹,

Zhao²,

Tu³

et al. 2023

Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

View full text Add to dashboard Cite

Molecular property prediction is an important problem in drug discovery and materials science. As geometric structures have been demonstrated necessary for molecular property prediction, 3D information has been combined with various graph learning methods to boost prediction performance. However, obtaining the geometric structure of molecules is not feasible in many real-world applications due to the high computational cost. In this work, we propose a novel 3D pre-training framework (dubbed 3D PGT), which pretrains a model on 3D molecular graphs, and then fine-tunes it on molecular graphs without 3D structures. Based on fact that bond length, bond angle, and dihedral angle are three basic geometric descriptors corresponding to a complete molecular 3D conformer, we first develop a multi-task generative pre-train framework based on these three attributes. Next, to automatically fuse these three generative tasks, we design a surrogate metric using the total energy to search for weight distribution of the three pretext tasks since total energy corresponding to the quality of 3D conformer. Extensive experiments on 2D molecular graphs are conducted to demonstrate the accuracy, efficiency and generalization ability of the proposed 3D PGT compared to various pre-training baselines. CCS CONCEPTS• Computing methodologies → Neural networks; • Applied computing → Molecular structural biology.

show abstract

Section: Pre-training On Large-scale Datasetmentioning

confidence: 99%

Automated 3D Pre-Training for Molecular Property Prediction

Wang¹,

Zhao²,

Tu³

et al. 2023

Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

View full text Add to dashboard Cite

show abstract

“…The typical form of world knowledge is a knowledge graph. Many works ( [24,25,26]) integrate entity and relation embedding from knowledge graph in pre-trained language models. WKLM [27] replaced entity mentions in the original documents with names of other entities of the same type and train the models to distinguish the correct entity mention from randomly chosen ones.…”

Section: Figurementioning

confidence: 99%

“…Experiments on a wide range of deep representation learning tasks show that UFO achieves higher accuracy with the smaller trimmed model than the single-task-Human Feedback) to reduce harmful content generation [24], expert labeling and high-quality cleaning of medical datasets to enhance the professional medical knowledge reserve of the model. Figure 7 Each of them learns the long-range relationships between multibody of the same order, and information of different orders can also be transferred across the tracks to enhance the modeling effect [25].…”

Section: Ernie 30mentioning

confidence: 99%

Domestic Large Model Technology and Medical Applications Analysis

Zhang¹,

Yu²

2023

Advanced Ultrasound in Diagnosis and Therapy

View full text Add to dashboard Cite

In 2023, the capabilities of text communication, language translation, image generation, and code writing demonstrated by ChatGPT and GPT-4 have received widespread attention in and out of China. With years of data and technology accumulation, many Chinese companies and research teams continue to make efforts in the field of large models, and their models cover many industries and have a number of characteristic functions. This paper will introduce the type and development trends of large models, and sort out the methods and characteristics of domestic large models. Finally, we will explain the advantages and disadvantages of the domestic models, and analyze the application prospect and challenges of them in the medical field.

show abstract

“…In addition to atom-level and bond-level graphs, Lu et al 32 also considered triple-level graphs to further extract multilevel representations of molecules. Liu et al 33 generalized multilevel representations as many-body interactions in molecules and designed the GEM-2 model which can capture M-body (where M can be 1, 2, 3, ...) level interactions through axial attentions. To incorporate prior chemical knowledge into GNNs, some studies take advantage of predefined functional group information to enhance molecular representations.…”

Section: ■ Introductionmentioning

confidence: 99%

Enhancing Molecular Representations Via Graph Transformation Layers

Ren

2023

J. Chem. Inf. Model.

View full text Add to dashboard Cite

Molecular representation learning is an essential component of many molecule-oriented tasks, such as molecular property prediction and molecule generation. In recent years, graph neural networks (GNNs) have shown great promise in this area, representing a molecule as a graph composed of nodes and edges. There are increasing studies showing that coarse-grained or multiview molecular graphs are important for molecular representation learning. Most of their models, however, are too complex and lack flexibility in learning different granular information for different tasks. Here, we proposed a flexible and simple graph transformation layer (i.e., LineEvo), a plug-and-use module for GNNs, which enables molecular representation learning from multiple perspectives. The LineEvo layer transforms fine-grained molecular graphs into coarse-grained ones based on the line graph transformation strategy. Especially, it treats the edges as nodes and generates the new connected edges, atom features, and atom positions. By stacking LineEvo layers, GNNs can learn multilevel information, from atom-level to triple-atoms level and coarser level. Experimental results show that the LineEvo layers can improve the performance of traditional GNNs on molecular property prediction benchmarks on average by 7%. Additionally, we show that the LineEvo layers can help GNNs have more expressive power than the Weisfeiler-Lehman graph isomorphism test.

show abstract

GEM-2: Next Generation Molecular Property Prediction Network by Modeling Full-range Many-body Interactions

Cited by 6 publications

References 17 publications

Automated 3D Pre-Training for Molecular Property Prediction

Automated 3D Pre-Training for Molecular Property Prediction

Domestic Large Model Technology and Medical Applications Analysis

Enhancing Molecular Representations Via Graph Transformation Layers

Contact Info

Product

Resources

About