DrugOOD: Out-of-Distribution (OOD) Dataset Curator and Benchmark for AI-aided Drug Discovery -- A Focus on Affinity Prediction Problems with Noise Annotations

Ji, Yuanfeng; Zhang, Lu; Wu, Jiaxiang; Wu, Bingzhe; Huang, Long-Kai; Xu, Tingyang; Yu, Rong; Li, Lanqing; Ren, Jie; Duan, Xue; Lai, Houtim; Shu, Xu; Feng, Jing; Liu, Wei; Luo, Ping; Zhou, Shuigeng; Huang, Junzhou; Zhao, Peilin; Bian, Yatao

doi:10.48550/arxiv.2201.09637

Cited by 17 publications

(36 citation statements)

References 124 publications

(164 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Ding et al [7] collect several datasets to compare the performance of well-known baselines and data augmentation methods. DrugOOD [19] is a recent benchmark specifically designed for molecular graph OOD problems. It is curated based on a large-scale bioassay dataset ChEMBL [31] and includes an automated pipeline for obtaining more datasets.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

GOOD: A Graph Out-of-Distribution Benchmark

Gui¹,

Li²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Out-of-distribution (OOD) learning deals with scenarios in which training and test data follow different distributions. Although general OOD problems have been intensively studied in machine learning, graph OOD is only an emerging area of research. Currently, there lacks a systematic benchmark tailored to graph OOD method evaluation. In this work, we aim at developing an OOD benchmark, known as GOOD, for graphs specifically. We explicitly make distinctions between covariate and concept shifts and design data splits that accurately reflect different shifts. We consider both graph and node prediction tasks as there are key differences when designing shifts. Overall, GOOD contains 8 datasets with 14 domain selections. When combined with covariate, concept, and no shifts, we obtain 42 different splits. We provide performance results on 7 commonly used baseline methods with 10 random runs. This results in 294 dataset-model combinations in total. Our results show significant performance gaps between in-distribution and OOD settings. Our results also shed light on different performance trends between covariate and concept shifts by different methods. Our GOOD benchmark is a growing project and expects to expand in both quantity and variety of resources as the area develops. The GOOD benchmark can be accessed via https://github.com/divelab/GOOD/. * Equal contributions Preprint. Under review.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Although both general OOD problems and graph analysis [21,11,40,46,28] have been intensively studied, graph OOD is only an emerging area of research [44,43,53,5]. Some initial attempts have also been made to curate graph OOD benchmarks [19,7]. However, existing benchmarks lack in several aspects, as detailed in Section 2.…”

Section: Introductionmentioning

confidence: 99%

GOOD: A Graph Out-of-Distribution Benchmark

Gui¹,

Li²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…A critical step in drug discovery is to select compounds with high biological activity (Wallach et al, 2015;Li et al, 2021;Ji et al, 2022), diversity and satisfactory ADME (absorption, distribution, metabolism, and excretion) properties (Gimeno et al, 2019). As a result, virtual screening is typically a hierarchical filtering process with several necessary filters, e.g., first choosing the highly active compounds, then selecting diverse subsets from them, and finally excluding compounds that are bad for ADME.…”

Section: Compound Selection In Ai-aided Drug Discoverymentioning

confidence: 99%

Learning Set Functions Under the Optimal Subset Oracle via Equivariant Variational Inference

Ou¹,

Xu²,

Su³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Learning set functions becomes increasingly more important in many applications like product recommendation and compound selection in AI-aided drug discovery. The majority of existing works study methodologies of set function learning under the function value oracle, which, however, requires expensive supervision signals. This renders it impractical for applications with only weak supervisions under the Optimal Subset (OS) oracle, the study of which is surprisingly overlooked. In this work, we present a principled yet practical maximum likelihood learning framework, termed as EquiVSet 1 , that simultaneously meets the following desiderata of learning set functions under the OS oracle: i) permutation invariance of the set mass function being modeled; ii) permission of varying ground set; iii) fully differentiability; iv) minimum prior; and v) scalability. The main components of our framework involve: an energy-based treatment of the set mass function, DeepSet-style architectures to handle permutation invariance, mean-field variational inference, and its amortized variants. Although the framework is embarrassingly simple, empirical studies on three real-world applications (including Amazon product recommendation, set anomaly detection and compound selection for virtual screening) demonstrate that EquiVSet outperforms the baselines by a large margin.

show abstract

“…iii) The optimal transport plan obtained by MOT can be used to design new message passing schemes in graph. iv) The proposed method can be potentially extended for more challenging settings that needs advanced knowledge transfer techniques, such as the graph out of distribution learning [Ji et al, 2022]…”

Section: Ablation Studiesmentioning

confidence: 99%

Fine-Tuning Graph Neural Networks via Graph Topology induced Optimal Transport

Zhang¹,

Xiao²,

Huang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Recently, the pretrain-finetuning paradigm has attracted tons of attention in graph learning community due to its power of alleviating the lack of labels problem in many real-world applications. Current studies use existing techniques, such as weight constraint, representation constraint, which are derived from images or text data, to transfer the invariant knowledge from the pre-train stage to fine-tuning stage. However, these methods failed to preserve invariances from graph structure and Graph Neural Network (GNN) style models. In this paper, we present a novel optimal transport-based fine-tuning framework called GTOT-Tuning, namely, Graph Topology induced Optimal Transport fine-Tuning, for GNN style backbones. GTOT-Tuning is required to utilize the property of graph data to enhance the preservation of representation produced by fine-tuned networks. Toward this goal, we formulate graph local knowledge transfer as an Optimal Transport (OT) problem with a structural prior and construct the GTOT regularizer to constrain the fine-tuned model behaviors. By using the adjacency relationship amongst nodes, the GTOT regularizer achieves node-level optimal transport procedures and reduces redundant transport procedures, resulting in efficient knowledge transfer from the pre-trained models. We evaluate GTOT-Tuning on eight downstream tasks with various GNN backbones and demonstrate that it achieves state-of-theart fine-tuning performance for GNNs.

show abstract

DrugOOD: Out-of-Distribution (OOD) Dataset Curator and Benchmark for AI-aided Drug Discovery -- A Focus on Affinity Prediction Problems with Noise Annotations

Cited by 17 publications

References 124 publications

GOOD: A Graph Out-of-Distribution Benchmark

GOOD: A Graph Out-of-Distribution Benchmark

Learning Set Functions Under the Optimal Subset Oracle via Equivariant Variational Inference

Fine-Tuning Graph Neural Networks via Graph Topology induced Optimal Transport

Contact Info

Product

Resources

About