Large-Scale Modeling of Sparse Protein Kinase Activity Data

Abstract

Protein kinases are a protein family that plays an important role in several complex diseases such as cancer and cardiovascular and immunological diseases. Protein kinases have conserved ATP binding sites, which when targeted can lead to similar activities of inhibitors against different kinases. This can be exploited to create multitarget drugs. On the other hand, selectivity (lack of similar activities) is desirable in order to avoid toxicity issues. There is a vast amount of protein kinase activity data in the public domain, which can be used in many different ways. Multitask machine learning models are expected to excel for these kinds of data sets because they can learn from implicit correlations between tasks (in this case activities against a variety of kinases). However, multitask modeling of sparse data poses two major challenges: (i) creating a balanced train–test split without data leakage and (ii) handling missing data. In this work, we construct a protein kinase benchmark set composed of two balanced splits without data leakage, using random and dissimilarity-driven cluster-based mechanisms, respectively. This data set can be used for benchmarking and developing protein kinase activity prediction models. Overall, the performance on the dissimilarity-driven cluster-based split is lower than on random split-based sets for all models, indicating poor generalizability of models. Nevertheless, we show that multitask deep learning models, on this very sparse data set, outperform single-task deep learning and tree-based models. Finally, we demonstrate that data imputation does not improve the performance of (multitask) models on this benchmark set.

Poor Generalization by Current Deep Learning Models for Predicting Binding Affinities of Kinase Inhibitors

“…Instead of simply grouping all analogs together into a given training/test fold, one can instead cluster all molecules based on their similarity, then build a test set comprised of the most "distinct" member of each cluster (chemical scaffold). This strategy -described by as "neighbor splitting" or "scaffold splitting"[49,[56][57][58][59][60] -does not entirely eliminate the information leakage that underlies "Standard Split" and "Split by Inhibitor", but rather minimizes the value of the leaked information to best mimic a real-world scenario in which prospective inputs are not wholly independent from the training examples.…”

Descriptor-augmented machine learning for enzyme-chemical interaction predictions

“…The splitting strategy of the dataset reflects how the model would be used in real scenarios [ 27 , 31 , 32 ]. And it would greatly affect the credibility of the results because of issues such as data breaches.…”

QSPRpred: a Flexible Open-Source Quantitative Structure-Property Relationship Modelling Tool

“…The choice of how to split data can greatly influence our impression of future performance of the model [22,70]. QSPRpred supports any scikit-learn-style [71] data splitter class that has a method split(X,y) that yields for each split/fold the indices for each subset.…”

“…Therefore, PCM has inherent applicability challenges that combine those of single-task and multi-task modelling (i.e. dataset size [19,20], data balance [21,22], sparsity [23], but also unique featurization requirements that need to take the proteins themselves into account [19,20]).…”