Assessing the Performances of Protein Function Prediction Algorithms from the Perspectives of Identification Accuracy and False Discovery Rate

Yu, Chun; Li, Xiao Xu; Yang, Hong; Li, Ying Hong; Xue, Weiwei; Chen, Yu Zong; Lin, Tao; Zhu, Feng

doi:10.3390/ijms19010183

Cited by 33 publications

(14 citation statements)

References 122 publications

(152 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Conversion of sequences into vector revealed good results in both groups used for analysis ( Table 1, 2 and Supplementary Table S2). In protein study, protein sequence converted into feature vectors showed good performance in cases of SVM and KNN (57)(58)(59)(60). RF00174, RF00059, RF00504, RF00522 predicted better than others with minority classes like RF01054, RF00634, RF00380 ( Table 1 and 2).…”

Section: Discussionmentioning

confidence: 99%

A Novel Riboswitch Classification based on Imbalanced Sequences achieved by Machine Learning

Beyene

Ling

Ristevski

et al. 2020

Preprint

View full text Add to dashboard Cite

Riboswitch, a part of mRNA (50-250nt in length), has two main classes: aptamer and expression platform. One of the main challenges raised during the classification of riboswitch is imbalanced data.That is a circumstance in which the records of a dataset of one group are very small compared to the others. Such circumstances lead classifier to ignore minority group and emphasize on majority class, that resulting with a skewed classification. We considered sixteen riboswitch families, to be in accord with recent riboswitch classification work, that contain imbalanced dataset ranging from 4,826 instances (RF00174) to 39 (RF01051) instances. The dataset was divided into training and test set using new developed pipeline. From 5460 k-mers, 156 features were produced calculated based on CfsSubsetEval and BestFirst. Statistically tested result was significantly difference between balanced and imbalanced dataset (p < 0.05). Besides, each algorithm also showed a significant difference in sensitivity, specificity, accuracy, and macro F-score when used in both groups (p < 0.05). Several k-mers clustered from heat map were discovered to have biological functions and motifs at the different positions like interior loops, terminal loops and helices. They were validated to have a biological function and some are riboswitch Beyene et al.2 motifs. The analysis has discovered the importance of solving the challenges of majority bias analysis and overfitting. Presented results were generalized evaluation of both balanced and imbalanced models, which implies their ability of classifying novel riboswitches. The scientific community can use python source code at https://github.com/Seasonsling/riboswitch, which can contribute to the process of developing software packages. Author SummaryMachine learning application has been used in many ways in bioinformatics and computational biology. Its use in riboswitch classification is still limited and existing attempt showed challenges due to imbalanced dataset. Algorithms classify dataset with majority and minority group, but they tend to ignore minority group and emphasize on majority class, consequential return a skewed classification We used new pipeline including SMOTE for balancing datasets that showed better classified riboswitch as well as improved performance of algorithms selected. Statistically significant difference observed between balanced and imbalanced in sensitivity, specificity, accuracy and F-score, this proved balanced dataset better for classification of riboswitch. Biological functions and motif search of k-mers in riboswitch families revealed their presence in interior loops, terminal loops and helices, some of the kmers were reported to be riboswitch motifs of aptamer domains and critical for metabolite binding. The pipeline can be used in machine learning and deep learning study in other domains of bioinformatics and computational biology suffering from imbalanced dataset. Finally, scientific community can use python source code, the work done and flow to develop packages.

show abstract

Section: Discussionmentioning

confidence: 99%

A Novel Riboswitch Classification based on Imbalanced Sequences achieved by Machine Learning

Beyene

Ling

Ristevski

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…This particular event was also observed in the relaxed condition of gene cluster detection for functional classification (Ling, et al, 2009). It has been also shown that ML efficiently assigns protein function (Yu, et al, 2018) and predicts the gene cluster that holds functional coupling (Chuang, et al, 2012). In this study, we applied the ML approach with gene neighborhood-based feature to facilitate the photosynthetic function assignment.…”

Section: Discussionmentioning

confidence: 99%

Photosynthetic protein classification using genome neighborhood-based machine learning feature

Sangphukieo

Laomettachit

Ruengjitchatchawalya

2020

Preprint

View full text Add to dashboard Cite

AbstractIdentification of novel photosynthetic proteins is important for understanding and improving photosynthetic efficiency. Synergistically, genomic context such as genome neighborhood can provide additional useful information to identify the photosynthetic proteins. We, therefore, expected that applying the computational approach, particularly machine learning (ML) with the genome neighborhood-based feature should facilitate the photosynthetic function assignment. Our results revealed a functional relationship between photosynthetic genes and their genomic neighbors, indicating the possibility to assign functions from their genome neighborhood profile. Therefore, we created a new method for extracting the patterns based on genome neighborhood network (GNN) and applied for the photosynthetic protein classification using ML algorithms. Random forest (RF) classifier using genome neighborhood-based features achieved the highest accuracy up to 94% in the classification of photosynthetic proteins and also showed better performance (Mathew’s correlation coefficient = 0.852) than other available tools including the sequence similarity search (0.497) and ML-based method (0.512). Furthermore, we demonstrated the ability of our model to identify novel photosynthetic proteins comparing to the other methods. Our classifier is available at http://bicep.kmutt.ac.th/photomod_standalone, https://bit.ly/2S0I2Ox and DockerHub: https://hub.docker.com/r/asangphukieo/photomod

show abstract

“…Diverse methods for proteomic data processing (transformation, normalization, and imputation) profoundly affected the precision of protein quantification which was frequently assessed using the value of pooled intragroup median absolute deviation (PMAD) of reported protein intensity among replicates ( Chawade et al, 2014 ; Kuharev et al, 2015 ; Valikangas et al, 2018 ; Yu et al, 2018 ). Particularly, the PMAD was designed to demonstrate the capacity of each analysis chain to reduce the variation among replicates, and therefore to enhance the technical reproducibility ( Chawade et al, 2014 ).…”

Section: Methodsmentioning

confidence: 99%

Discovery of the Consistently Well-Performed Analysis Chain for SWATH-MS Based Pharmacoproteomic Quantification

Tang

Wang

et al. 2018

Front. Pharmacol.

Self Cite

View full text Add to dashboard Cite

Sequential windowed acquisition of all theoretical fragment ion mass spectra (SWATH-MS) has emerged as one of the most popular techniques for label-free proteome quantification in current pharmacoproteomic research. It provides more comprehensive detection and more accurate quantitation of proteins comparing with the traditional techniques. The performance of SWATH-MS is highly susceptible to the selection of processing method. Till now, ≥27 methods (transformation, normalization, and missing-value imputation) are sequentially applied to construct numerous analysis chains for SWATH-MS, but it is still not clear which analysis chain gives the optimal quantification performance. Herein, the performances of 560 analysis chains for quantifying pharmacoproteomic data were comprehensively assessed. Firstly, the most complete set of the publicly available SWATH-MS based pharmacoproteomic data were collected by comprehensive literature review. Secondly, substantial variations among the performances of various analysis chains were observed, and the consistently well-performed analysis chains (CWPACs) across various datasets were for the first time generalized. Finally, the log and power transformations sequentially followed by the total ion current normalization were discovered as one of the best performed analysis chains for the quantification of SWATH-MS based pharmacoproteomic data. In sum, the CWPACs identified here provided important guidance to the quantification of proteomic data and could therefore facilitate the cutting-edge research in any pharmacoproteomic studies requiring SWATH-MS technique.

show abstract

Assessing the Performances of Protein Function Prediction Algorithms from the Perspectives of Identification Accuracy and False Discovery Rate

Cited by 33 publications

References 122 publications

A Novel Riboswitch Classification based on Imbalanced Sequences achieved by Machine Learning

A Novel Riboswitch Classification based on Imbalanced Sequences achieved by Machine Learning

Photosynthetic protein classification using genome neighborhood-based machine learning feature

Discovery of the Consistently Well-Performed Analysis Chain for SWATH-MS Based Pharmacoproteomic Quantification

Contact Info

Product

Resources

About