Machine learning for discovering missing or wrong protein function annotations

Abstract

Background A massive amount of proteomic data is generated on a daily basis, nonetheless annotating all sequences is costly and often unfeasible. As a countermeasure, machine learning methods have been used to automatically annotate new protein functions. More specifically, many studies have investigated hierarchical multi-label classification (HMC) methods to predict annotations, using the Functional Catalogue (FunCat) or Gene Ontology (GO) label hierarchies. Most of these studies employed benchmark datasets created more than a decade ago, and thus train their models on outdated information. In this work, we provide an updated version of these datasets. By querying recent versions of FunCat and GO yeast annotations, we provide 24 new datasets in total. We compare four HMC methods, providing baseline results for the new datasets. Furthermore, we also evaluate whether the predictive models are able to discover new or wrong annotations, by training them on the old data and evaluating their results against the most recent information. Results The results demonstrated that the method based on predictive clustering trees, Clus-Ensemble, proposed in 2008, achieved superior results compared to more recent methods on the standard evaluation task. For the discovery of new knowledge, Clus-Ensemble performed better when discovering new annotations in the FunCat taxonomy, whereas hierarchical multi-label classification with genetic algorithm (HMC-GA), a method based on genetic algorithms, was overall superior when detecting annotations that were removed. In the GO datasets, Clus-Ensemble once again had the upper hand when discovering new annotations, HMC-GA performed better for detecting removed annotations. However, in this evaluation, there were less significant differences among the methods. Conclusions The experiments have showed that protein function prediction is a very challenging task which should be further investigated. We believe that the baseline results associated with the updated datasets provided in this work should be considered as guidelines for future studies, nonetheless the old versions of the datasets should not be disregarded since other tasks in machine learning could benefit from them.

Systems biology approaches integrated with artificial intelligence for optimized metabolic engineering

“…An early attempt of using ML in genes functional annotation from biomedical literature utilized Hierarchical Text Categorization (HTC) [ 98 ], while Tetko et al provided a high-quality curated functional annotation data as a benchmark dataset for the developers of machine ML-based functional annotation methods for bacterial genomes [ 99 ]. The recent reports show the applications of ML-based methods in a wide variety of functional annotations such as the discovery of missing or wrong protein function annotations [ 100 ], predicting gene functions in plant [ 101 ], controlling the false discovery rate (FDR), increase the accuracy of protein functional predictions [ 102 ], and genome-wide functional annotation of splice-variants in eukaryotes [ 103 ].…”

Hierarchical multilabel classifier for gene ontology annotation using multihead and multiend deep CNN model

“…In this paper, we only used the hierarchy GO dataset up to five levels. In our future work, we will apply the proposed model to GO datasets with more levels . Moreover, we will carry out a study on applying other deep learning models, especially transfer learning for protein function predictions.…”

Feature Selection for Hierarchical Multi-label Classification

“…We used ten HMC protein function datasets modeled as a tree. They are commonly used to evaluate hierarchical multi-label classifiers [9], and are freely available 1 . They come with already prepared training, validation and test partitions, which have been used by many works in the literature.…”