Adding stochastic negative examples into machine learning improves molecular bioactivity prediction

Abstract

Multitask deep neural networks learn to predict ligand-target binding by example, yet public pharmacological datasets are sparse, imbalanced, and approximate. We constructed two hold-out benchmarks to approximate temporal and drug-screening test scenarios whose characteristics differ from a random split of conventional training datasets. We developed a pharmacological dataset augmentation procedure, Stochastic Negative Addition (SNA), that randomly assigns untested molecule-target pairs as transient negative examples during training. Under the SNA procedure, ligand drug-screening benchmark performance increases from R 2 = 0.1926 ± 0.0186 to 0.4269±0.0272 (121.7%). This gain was accompanied by a modest decrease in the temporal benchmark (13.42%). SNA increases in drug-screening performance were consistent for classification and regression tasks and outperformed scrambled controls. Our results highlight where data and feature uncertainty may be problematic, but also show how leveraging uncertainty into training improves predictions of drug-target relationships.

Comparing Fingerprints for Ligand-Based Virtual Screening: A Fast and Scalable Approach for Unbiased Evaluation

“…We used cross-validation as a comparative procedure to compare the performance of the CATS and Morgan fingerprints, both of which incurred the same amount of data loss, and we did not attempt to estimate the true prospective performance rate. Furthermore, optimal performance is reached when subsampling the negative class (as we did) rather than using as much negative data as available. , …”

“…Furthermore, optimal performance is reached when subsampling the negative class (as we did) rather than using as much negative data as available. 29,30 Monte Carlo cross-validation differs from k-fold crossvalidation in that the test set in each case is sampled at random (without replacement) from the entire data set, as opposed to being randomly partitioned once. In the limit of infinite evaluations, this guarantees that all instances will eventually be evaluated, and with different combinations of the test data set, which reduces variance compared with k-fold cross-validation.…”

“…We preferred the approach of adding presumed-negatives over subsampling the positive class as the classifier performance degrades in proportion to the amount of removed positives . We did not explicitly optimize for the ratio of inactives to actives (10:1 has been proposed as the optimal for a data set with orders of magnitude smaller than ours , ). The Monte Carlo subsampling can easily be adjusted to achieve any desired ratio.…”

Prioritizing virtual screening with interpretable interaction fingerprints

“…In contrast, long FPs cause compute performance and storage issues and may produce overfitted ML models, a problem known as the curse of dimensionality. 84 Several authors explored different MFP and IFP lengths, 7, 59, 60, 71, 85–95 however there is no single consensus on length, as results vary with the problem and data sets.…”

Putting Density Functional Theory to the Test in Machine-Learning-Accelerated Materials Discovery

“…88%) of the feasible chemical space is explored in 33% of the computational time (Table 1). 37 and materials discovery [38][39] ). However, the overhead of up-front data generation is overshadowed by the concern of transferability.…”