Effect of Dimensionality Reduction on Classification Accuracy for Protein–Protein Interaction Prediction

Mahapatra, Soumya Jagannath; Kumar, Anish; Sharma, Animesh; Sahu, Sitanshu Sekhar

doi:10.1007/978-981-15-1081-6_1

Cited by 2 publications

(2 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since we only used default hyperparameters, we did not need a validation set and added the validation to training for these two datasets. We chose the random 80/20 split since most reviewed methods report the mean accuracy of five-fold cross-validation 4-8, 11, 18, 33, 36, 37 or a random hold-out test set 2,3,12,13,32,[38][39][40][41] .…”

Section: Datasetmentioning

confidence: 99%

“…D-SCRIPT and Topsy-Turvy only reported auPR and AUC on their dataset. We chose the random 80/20 split since most reviewed methods report the mean accuracy of five-fold cross-validation 4-8, 11, 18, 38, 41, 42 or a random hold-out test set 2,3,12,13,37,[43][44][45][46] . As many datasets are rather small, those models which were developed for larger datasets (e.g., D-SCRIPT and Topsy-Turvy) could be prone to overfitting.…”

Section: Overviewmentioning

confidence: 99%

See 1 more Smart Citation

Cracking the black box of deep sequence-based protein-protein interaction prediction

Bernett

Blumenthal

List

2023

Preprint

View full text Add to dashboard Cite

Identifying protein-protein interactions (PPIs) is crucial for deciphering biological pathways and their dysregulation. Numerous prediction methods have been developed as a cheap alternative to biological experiments, reporting phenomenal accuracy estimates. While most methods rely exclusively on sequence information, PPIs occur in 3D space. As predicting protein structure from sequence is an infamously complex problem, the almost perfect reported performances for PPI prediction seem dubious. We systematically investigated how much reproducible deep learning models depend on data leakage, sequence similarities, and node degree information and compared them to basic machine learning models. We found that overlaps between training and test sets resulting from random splitting lead to strongly overestimated performances, giving a false impression of the field. In this setting, models learn solely from sequence similarities and node degrees. When data leakage is avoided by minimizing sequence similarities between training and test, performances become random, leaving this research field wide open.

show abstract

Section: Datasetmentioning

confidence: 99%

Section: Overviewmentioning

confidence: 99%

Cracking the black box of deep sequence-based protein-protein interaction prediction

Bernett

Blumenthal

List

2023

Preprint

View full text Add to dashboard Cite

show abstract

Cracking the black box of deep sequence-based protein–protein interaction prediction

Bernett,

Blumenthal,

List

2024

Briefings in Bioinformatics

View full text Add to dashboard Cite

Identifying protein–protein interactions (PPIs) is crucial for deciphering biological pathways. Numerous prediction methods have been developed as cheap alternatives to biological experiments, reporting surprisingly high accuracy estimates. We systematically investigated how much reproducible deep learning models depend on data leakage, sequence similarities and node degree information, and compared them with basic machine learning models. We found that overlaps between training and test sets resulting from random splitting lead to strongly overestimated performances. In this setting, models learn solely from sequence similarities and node degrees. When data leakage is avoided by minimizing sequence similarities between training and test set, performances become random. Moreover, baseline models directly leveraging sequence similarity and network topology show good performances at a fraction of the computational cost. Thus, we advocate that any improvements should be reported relative to baseline methods in the future. Our findings suggest that predicting PPIs remains an unsolved task for proteins showing little sequence similarity to previously studied proteins, highlighting that further experimental research into the ‘dark’ protein interactome and better computational methods are needed.

show abstract

Effect of Dimensionality Reduction on Classification Accuracy for Protein–Protein Interaction Prediction

Cited by 2 publications

References 15 publications

Cracking the black box of deep sequence-based protein-protein interaction prediction

Cracking the black box of deep sequence-based protein-protein interaction prediction

Cracking the black box of deep sequence-based protein–protein interaction prediction

Contact Info

Product

Resources

About