Attention-wise masked graph contrastive learning for predicting molecular property

Abstract

Motivation Accurate and efficient prediction of the molecular property is one of the fundamental problems in drug research and development. Recent advancements in representation learning have been shown to greatly improve the performance of molecular property prediction. However, due to limited labeled data, supervised learning-based molecular representation algorithms can only search limited chemical space and suffer from poor generalizability. Results In this work, we proposed a self-supervised learning method, ATMOL, for molecular representation learning and properties prediction. We developed a novel molecular graph augmentation strategy, referred to as attention-wise graph masking, to generate challenging positive samples for contrastive learning. We adopted the graph attention network as the molecular graph encoder, and leveraged the learned attention weights as masking guidance to generate molecular augmentation graphs. By minimization of the contrastive loss between original graph and augmented graph, our model can capture important molecular structure and higher order semantic information. Extensive experiments showed that our attention-wise graph mask contrastive learning exhibited state-of-the-art performance in a couple of downstream molecular property prediction tasks. We also verified that our model pretrained on larger scale of unlabeled data improved the generalization of learned molecular representation. Moreover, visualization of the attention heatmaps showed meaningful patterns indicative of atoms and atomic groups important to specific molecular property.

ToxMPNN: A deep learning model for small molecule toxicity prediction

“…Traditional MPNN consists of two stages: the message passing stage and the aggregation stage. To address the challenge of extracting important information from the complex molecular graph structure (Liu, Huang, Liu, & Deng, 2022), we have designed a novel method called ToxMPNN. This work can be divided into three parts (molecular features construction, aggregation and combination function, and readout function) in Figure 3.…”

“…For TextCNN, only the SMILES strings are needed to be the inputs.2.3 | ToxMPNN model constructionTraditional MPNN consists of two stages: the message passing stage and the aggregation stage. To address the challenge of extracting important information from the complex molecular graph structure(Liu, Huang, Liu, & Deng, 2022), we have designed a novel method called ToxMPNN. This work can be divided into three parts (molecular features construction, aggregation and combination function, and F I G U R E 2 Comparison of the positive and negative sample counts for the seven toxicity categories in the Toxicity_drug dataset.T A B L E 1 Models and corresponding features and datasets used in this paper.…”

Attention is all you need: utilizing attention in AI-enabled drug discovery

“…Liu et al. [ 57 ] proposed an attention-wise graph masking strategy, utilizing GAT as a molecular graph encoder and the learned attention weights as masking guides to generate enhanced molecular graphs for property prediction. Wang et al.…”

“…FraGAT, a fragment-oriented multi-scale graph attention model, excels in capturing diverse views of molecule features, especially emphasizing functional groups that play a crucial role in a molecule’s properties [ 139 ]. ATMOL , on the other hand, introduces attention-wise graph masking, significantly enhancing molecular representation and consequently, downstream molecular property prediction tasks [ 57 ]. FP-GNN represents a notable stride by effectively combining information from molecular graphs and fingerprints, resulting in precise molecular property prediction [ 140 ].…”

Leveraging transformers‐based language models in proteome bioinformatics

“…A major obstacle in drug discovery is to recognize promising drug ing SMILES-BERT [86], GROVER [87], and KPGT [88]. Furthermore, Liu et al have presented attention-wise masked graph contrastive learning to predict molecular properties [89]. Additionally, DeepR2cov is a promising approach for heterogeneous drug network deep representation learning, which has the potential to identify anti-inflammatory agents for the treatment of COVID-19 [90].…”