A large-scale evaluation of computational protein function prediction

Radivojac, Predrag; Clark, Wyatt T.; Oron, Tal; Schnoes, Alexandra M.; Wittkop, Tobias; Sokolov, Artem; Graim, Kiley; Funk, Christopher; Verspoor, Karin; Ben‐Hur, Asa; Pandey, Gaurav; Yunes, Jeffrey M.; Talwalkar, Ameet; Repo, Susanna; Souza, Michael L; Piovesan, Damiano; Casadio, Rita; Wang, Zheng; Cheng, Jianlin; Fang, Hai; Gough, Julian; Koskinen, Patrik; Törönen, Petri; Nokso-Koivisto, Jussi; Holm, Liisa; Cozzetto, Domenico; Buchan, Daniel W. A.; Bryson, Kevin; Jones, David T.; Limaye, Bhakti; Inamdar, Harshal; Datta, Avik; Manjari, Sunitha K; Joshi, Rajendra; Chitale, Meghana; Kihara, Daisuke; Lisewski, Andreas Martin; Erdin, Serkan; Venner, Eric; Lichtarge, Olivier; Rentzsch, Robert; Yang, Haixuan; Romero, Alfonso E.; Bhat, Prajwal; Paccanaro, Alberto; Hamp, Tobias; Kaßner, Rebecca; Seemayer, Stefan; Vicedo, Esmeralda; Schaefer, Christian; Achten, Dominik; Auer, Florian; Boehm, Ariane C; Braun, Tatjana; Hecht, Maximilian; Heron, Mark; Hönigschmid, Peter; Hopf, Thomas A.; Kaufmann, Stefanie; Kiening, Michael; Krompaß, Denis; Landerer, Cedric; Mahlich, Yannick; Roos, Manfred; Björne, Jari; Salakoski, Tapio; Wong, Andrew; Shatkay, Hagit; Gatzmann, Fanny; Sommer, Iris E.; Wass, Mark N.; Sternberg, Michael J.E.; Škunca, Nives; Supek, Fran; Bošnjak, Matko; Panov, Panče; Džeroski, Sašo; Šmuc, Tomislav; Kourmpetis, Yiannis A. I.; Dijk, Aalt D. J. van; Braak, Cajo J. F. ter; Zhou, Yuanpeng; Gong, Qingtian; Dong, Xinran; Tian, Weidong; Falda, Marco; Fontana, Paolo; Lavezzo, Enrico; Camillo, Barbara Di; Toppo, Stefano; Liang, Liuqin; Djuric, Nemanja; Guo, Yuhong; Vučetić, Slobodan; Bairoch, Amos Marc; Linial, Michal; Babbitt, Patricia C.; Brenner, Steven E.; Orengo, Christine A.; Rost, Burkhard; Mooney, Sean D.; Friedberg, Iddo

doi:10.1038/nmeth.2340

Cited by 888 publications

(1,025 citation statements)

References 52 publications

Supporting

Mentioning

984

Contrasting

Unclassified

Order By: Relevance

“…We applied our algorithm to the Automated protein Function Prediction (AFP) problem, a challenging and central problem in computational biology [53,50]. In this setting, nodes represent proteins and connections their pairwise relationships deriving from different sources of information, including gene co-expression, genetic and physical interactions, protein ontologies and phenotype annotations.…”

Section: Methodsmentioning

confidence: 99%

“…We compared our method with several state-of-the-art competitors: GBA, an algorithm based on the guilt-by-association principle [39]; GeneMANIA [44], the top method in the MouseFunc challenge [52]; MS-kNN [32], one of the top ranking algorithm in the recent CAFA challenge [53]; LP, a semisupervised label propagation algorithm based on Gaussian random fields, and its class mass normalized version LP-CMN [70]; RW, the classical random walk algorithm without restart with at most 1000 random walk steps [36]; COSNet, a recently proposed algorithm for label prediction in graph, which is explicitly designed to cope with the imbalance in the instance labeling [21]. Furthermore, to assess the improvements introduced by partitioning proteins into categories, we also apply a different version of HoMCat, in which all the proteins are considered belonging to the same category (named HoMCat-1c).…”

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 99%

“…when positive examples are significantly fewer than those negative, and imbalance-aware strategies are required [16,67,35,34]. Unfortunately, several graph-based prediction problems are characterized by strongly unbalanced labelings [13,53,21].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Learning node labels with multi-category Hopfield networks

Frasca

Bassis

Valentini

2015

Neural Comput & Applic

View full text Add to dashboard Cite

In several real-world node-label prediction problems on graphs, in fields ranging from computational biology to World-Wide-Web analysis, nodes can be partitioned into categories different from the classes to be predicted, on the basis of their characteristics or their common properties. Such partitions may provide further information about node classification that classical machine learning algorithms do not take into account. We introduce a novel family of parametric Hopfield networks (m-Category Hopfield Networks) and a novel algorithm (Hopfield Multi-Category -HoMCat), designed to appropriately exploit the presence of propertybased partitions of nodes into multiple categories. Moreover, the proposed model adopts a cost-sensitive learning strategy to prevent the remarkable decay in performance usually observed when instance labels are unbalanced, that is when one class of labels is highly under-represented than the other one. We validate the proposed model on both synthetic and real-world data, in the context of multi-species function prediction, where the classes to be predicted are the Gene Ontology terms and the categories the different species in the multi-species protein network. We carried out an intensive experimental validation, which on the one hand compares HoMCat with several state-of-the-art graph-based algorithms, and on the other hand reveals that exploiting meaningful prior partitions of input data can substantially improve classification performances.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Learning node labels with multi-category Hopfield networks

Frasca

Bassis

Valentini

2015

Neural Comput & Applic

View full text Add to dashboard Cite

show abstract

“…These include sequence similarity [27,28] Among these in-silico methods [52], the basic local alignment search tool (BLAST) [53] revealing protein functions based on excess sequence similarity [54] demonstrated great capacity and attracted substantial interests from the researchers of this field [55,56]. Apart from BLAST, the methods based on the machine learning algorithm (a specific type of artificial intelligence) were frequently used in recent years to predict protein function [57][58][59][60][61][62], and various types of software together with several web-based tools integrating these methods were developed to predict the protein function from sequences irrespective of sequence or structural similarity [36,63].…”

Section: Introductionmentioning

confidence: 99%

Assessing the Performances of Protein Function Prediction Algorithms from the Perspectives of Identification Accuracy and False Discovery Rate

Yu¹,

Li²,

Yang³

et al. 2017

Preprint

View full text Add to dashboard Cite

The knowledge of protein function is essential for the study of biological processes, the understanding of disease mechanism and the exploration of novel therapeutic target. Apart from experimental methods, a number of in-silico approaches have been developed and extensively used for protein function prediction.Among these approaches, BLAST predicts functions based on protein sequence similarity, and machine learning predicts functional families from protein sequences irrespective of their similarity, which complements BLAST and other methods in predicting diverse classes of proteins including distantly related proteins and homologous proteins of different functions. However, their identification accuracies and the false discovery rate have not yet been assessed so far, which greatly limits the usage of these prediction algorithms. Herein, a comprehensive comparison of the performances among four popular functional prediction algorithms (BLAST, SVM, PNN and KNN) was conducted. In particular, the performance of these algorithms were systematically assessed by four metrics (sensitivity, specificity, accuracy and Matthews correlation coefficient) based on the independent test datasets generated from 93 protein families defined by UniProtKB Keywords. Moreover, the false discovery rates of these algorithms were evaluated by scanning the genomes of four representative model species (homo sapiens, arabidopsis thaliana, saccharomyces cerevisiae and mycobacterium tuberculosis). As a result, the substantially higher sensitivity and stability of BLAST and SVM were observed compared with that of PNN and KNN. But the machine learning algorithms (PNN, KNN and SVM) were found capable of significantly reducing the false discovery rate (SVM < PNN ≈ KNN). In summary, this study comprehensively assessed the performance of four popular algorithms applied to protein function prediction, which could facilitate the selection of the most appropriate method in the related biomedical research. KEYWORDSfalse discovery rate; machine learning; protein function prediction; support vector machine; BLAST Preprints (www.preprints.org) | NOT PEER-REVIEWED | Posted:

show abstract

“…ML approaches are the state of the art in most non-classic prediction challenges. These methods are applied in community annotation challenges such as Critical Assessment of protein Function Annotation (CAFA) (5,6), and Critical Assessment for Information Extraction in Biology (BioCreAtIvE) (7). ML approaches actually benefit from the growth of available sequences, while 'brittle' rulebased methods often fail to cope with the growing variability and quantity of possible annotations and sequences.…”

Section: Introductionmentioning

confidence: 99%

ASAP: A Machine-Learning Framework for Local Protein Properties

Ofer

Brandes

Linial

2015

Preprint

Self Cite

View full text Add to dashboard Cite

Determining residue-level protein properties, such as sites of post-translational modifications (PTMs), is vital to understanding protein function. Experimental methods are costly and time-consuming, while traditional rule-based computational methods fail to annotate sites lacking substantial similarity. Machine Learning (ML) methods are becoming fundamental in annotating unknown proteins and their heterogeneous properties. We present ASAP (Amino-acid Sequence Annotation Prediction), a universal ML framework for predicting residue-level properties. ASAP extracts numerous features from raw sequences, and supports easy integration of external features such as secondary structure, solvent accessibility, intrinsically disorder or PSSM profiles. Features are then used to train ML classifiers. ASAP can create new classifiers within minutes for a variety of tasks, including PTM prediction (e.g. cleavage sites by convertase, phosphoserine modification). We present a detailed case study for ASAP: CleavePred, an ASAP-based model to predict protein precursor cleavage sites, with state-of-the-art results. Protein cleavage is a PTM shared by a wide variety of proteins sharing minimal sequence similarity. Current rule-based methods suffer from high false positive rates, making them suboptimal. The high performance of CleavePred makes it suitable for analyzing new proteomes at a genomic scale. The tool is attractive to protein design, mass spectrometry search engines and the discovery of new bioactive peptides from precursors. ASAP functions as a baseline approach for residue-level protein sequence prediction. CleavePred is freely accessible as a web-based application. Both ASAP and CleavePred are open-source with a flexible Python API.Database URL: ASAP's and CleavePred source code, webtool and tutorials are available at:

show abstract

A large-scale evaluation of computational protein function prediction

Cited by 888 publications

References 52 publications

Learning node labels with multi-category Hopfield networks

Learning node labels with multi-category Hopfield networks

Assessing the Performances of Protein Function Prediction Algorithms from the Perspectives of Identification Accuracy and False Discovery Rate

ASAP: A Machine-Learning Framework for Local Protein Properties

Contact Info

Product

Resources

About