DRG2vec: Learning Word Representations from Definition Relational Graph

Shu, Xiaobo; Yu, Bowen; Zhang, Zhenyu; Liu, Tingwen

doi:10.1109/ijcnn48605.2020.9207015

Cited by 10 publications

(17 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this study, we maintain the mask ratio of 15, 25, and 15% for AttrMask, GraphMAE, and MAM, respectively, as per their original experimental settings. 10,11,13 Since MAM is designed to tackle the problem of imbalanced atom distribution, we test our method only with AttrMask and GraphMAE in this study. It is essential to note that pretraining tasks with our proposed masking strategy retain the same settings as that of the previous works 10,13 and will be denoted as AttrMask(WMM) or GraphMAE(WMM) for clarity.…”

Section: Analysis Of Moleculesmentioning

confidence: 99%

“…10,11,13 Since MAM is designed to tackle the problem of imbalanced atom distribution, we test our method only with AttrMask and GraphMAE in this study. It is essential to note that pretraining tasks with our proposed masking strategy retain the same settings as that of the previous works 10,13 and will be denoted as AttrMask(WMM) or GraphMAE(WMM) for clarity. However, the pretraining methods with random masking will not be explicitly annotated in the subsequent sections.…”

Section: Analysis Of Moleculesmentioning

confidence: 99%

“…7−9 The MLM-style pretraining strategy is also utilized for pretraining graph neural networks (GNNs) to extract chemical knowledge from large-scale unlabeled molecular graph data sets, with one pioneering work known as Attribute Masking (AttrMask). 10 However, MOLE-BERT 11 highlights a critical issue with AttrMask, namely, the imbalanced distribution of atoms. As seen in Figure 1a, carbon atoms make up a significant portion, approximately 74%, of the total, while all trace elements account for only 4%.…”

Section: ■ Introductionmentioning

confidence: 99%

“…In the stage of pretraining, we utilize a data set obtained from the ZINC15 database, 15 which is widely used in previous works. 10,16 Then, we adopt eight data sets commonly used for binary classification from MoleculeNet 17 for transfer learning. The transfer learning centers on molecular property prediction.…”

Section: ■ Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Rethinking the Masking Strategy for Pretraining Molecular Graphs from a Data-Centric View

Lin,

Fung

2024

ACS Omega

View full text Add to dashboard Cite

Node-level self-supervised learning has been widely applied for pretraining molecular graphs. Attribute Masking (AttrMask) is pioneering work in this field, and its improved methods focus on enhancing the capacity of the backbone models by incorporating additional modules. However, these methods overlook the imbalanced atom distribution due to employing only the random masking strategy to mask atoms for pretraining. According to the properties of molecules, we propose a weighted masking strategy to enhance the capacity of pretrained models by more effective utilization of molecular information while pretraining. Our experimental results demonstrate that AttrMask combined with our proposed weighted masking strategy yields superior performance compared to the random masking strategy, even surpassing the model-centric improvement methods without increasing the parameters. Additionally, our weighted masking strategy can be extended to other pretraining methods to achieve enhanced performance.

show abstract

Section: Analysis Of Moleculesmentioning

confidence: 99%

Section: Analysis Of Moleculesmentioning

confidence: 99%

Section: ■ Introductionmentioning

confidence: 99%

Section: ■ Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Rethinking the Masking Strategy for Pretraining Molecular Graphs from a Data-Centric View

Lin,

Fung

2024

ACS Omega

View full text Add to dashboard Cite

show abstract

“…In fact, learned feature mapping (via deep neural network for non-tabular and structured data) allows for learning (namely generating) features customized for the problem at hand potentially reducing the number of features to the minimum necessary number. The difference, with respect to the case of learned feature mapping for tabular data, is that in the case of non-tabular data, the approaches for learned feature mapping are becoming quite effective and efficient thanks to the availability of huge amount of data that allows to pretrain model in one application that can be applied and fine-tuned to other problems [ 160 – 162 ]. These pretrained models map complex data structures into reasonably small cardinality yet quite expressive features that can be reused efficiently and effectively in many applications.…”

Section: Tip 8: Appropriately Feature Engineer Non-tabular Datamentioning

confidence: 99%

Eleven quick tips for data cleaning and feature engineering

2022

View full text Add to dashboard Cite

Applying computational statistics or machine learning methods to data is a key component of many scientific studies, in any field, but alone might not be sufficient to generate robust and reliable outcomes and results. Before applying any discovery method, preprocessing steps are necessary to prepare the data to the computational analysis. In this framework, data cleaning and feature engineering are key pillars of any scientific study involving data analysis and that should be adequately designed and performed since the first phases of the project. We call “feature” a variable describing a particular trait of a person or an observation, recorded usually as a column in a dataset. Even if pivotal, these data cleaning and feature engineering steps sometimes are done poorly or inefficiently, especially by beginners and unexperienced researchers. For this reason, we propose here our quick tips for data cleaning and feature engineering on how to carry out these important preprocessing steps correctly avoiding common mistakes and pitfalls. Although we designed these guidelines with bioinformatics and health informatics scenarios in mind, we believe they can more in general be applied to any scientific area. We therefore target these guidelines to any researcher or practitioners wanting to perform data cleaning or feature engineering. We believe our simple recommendations can help researchers and scholars perform better computational analyses that can lead, in turn, to more solid outcomes and more reliable discoveries.

show abstract

3D graph contrastive learning for molecular property prediction

Moon

Kwon

2023

Bioinformatics

View full text Add to dashboard Cite

Motivation Self-supervised learning (SSL) is a method that learns the data representation by utilizing supervision inherent in the data. This learning method is in the spotlight in the drug field, lacking annotated data due to time-consuming and expensive experiments. SSL using enormous unlabeled data has shown excellent performance for molecular property prediction, but a few issues exist. (1) Existing SSL models are large-scale; there is a limitation to implementing SSL where the computing resource is insufficient. (2) In most cases, they do not utilize 3D structural information for molecular representation learning. The activity of a drug is closely related to the structure of the drug molecule. Nevertheless, most current models do not use 3D information or use it partially. (3) Previous models that apply contrastive learning to molecules use the augmentation of permuting atoms and bonds. Therefore, molecules having different characteristics can be in the same positive samples. We propose a novel contrastive learning framework, small-scale 3D Graph Contrastive Learning (3DGCL) for molecular property prediction, to solve the above problems. Results 3DGCL learns the molecular representation by reflecting the molecule’s structure through the pre-training process that does not change the semantics of the drug. Using only 1,128 samples for pre-train data and 0.5 million model parameters, we achieved state-of-the-art or comparable performance in six benchmark datasets. Extensive experiments demonstrate that 3D structural information based on chemical knowledge is essential to molecular representation learning for property prediction. Availability Data and codes are available in https://github.com/moonkisung/3DGCL.

show abstract

DRG2vec: Learning Word Representations from Definition Relational Graph

Cited by 10 publications

References 26 publications

Rethinking the Masking Strategy for Pretraining Molecular Graphs from a Data-Centric View

Rethinking the Masking Strategy for Pretraining Molecular Graphs from a Data-Centric View

Eleven quick tips for data cleaning and feature engineering

3D graph contrastive learning for molecular property prediction

Contact Info

Product

Resources

About