Accuracy of protein-level disorder predictions

Katuwawala, Akila; Oldfield, Christopher J.; Kurgan, Lukasz

doi:10.1093/bib/bbz100

Cited by 41 publications

(51 citation statements)

References 124 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is apparent when reading the 'suggested best disorder predictor' and 'year the most recent assessed predictor published' columns in Table 1. A case in point is the most recent comparative survey [93] that concludes that SPOT-Disorder [74] and DISOPRED3 [57] outperform the other current tools, while SPOT-Disorder was released after all but two of the 11 comparative surveys were published. Considering the three most-recent empirical assessments [45,91,93], the best performing predictors include SPOT-Disorder [74], DISOPRED3 [57], ESpritz [60], DisEMBL [53] and IUPred [62][63][64].…”

Section: Surveys Of the Intrinsic Disorder Predictorsmentioning

confidence: 99%

“…For instance, the CASP assessments rely on unreleased depositions into the PDB that were not screened against the training proteins [85][86][87][88][89][90]. Similarly, the datasets used in the recent comparative studies [45,91,93] were collected from the MobiDB, DisProt and UniProt resources without screening them against the training datasets. Similarly, the forthcoming CAID assessment relies on the benchmark proteins collected from DisProt that were not screened for similarity to the training proteins [110].…”

Section: Surveys Of the Intrinsic Disorder Predictorsmentioning

confidence: 99%

See 1 more Smart Citation

Comparative Assessment of Intrinsic Disorder Predictions with a Focus on Protein and Nucleic Acid-Binding Proteins

Katuwawala

Kurgan

2020

Biomolecules

Self Cite

View full text Add to dashboard Cite

With over 60 disorder predictors, users need help navigating the predictor selection task. We review 28 surveys of disorder predictors, showing that only 11 include assessment of predictive performance. We identify and address a few drawbacks of these past surveys. To this end, we release a novel benchmark dataset with reduced similarity to the training sets of the considered predictors. We use this dataset to perform a first-of-its-kind comparative analysis that targets two large functional families of disordered proteins that interact with proteins and with nucleic acids. We show that limiting sequence similarity between the benchmark and the training datasets has a substantial impact on predictive performance. We also demonstrate that predictive quality is sensitive to the use of the well-annotated order and inclusion of the fully structured proteins in the benchmark datasets, both of which should be considered in future assessments. We identify three predictors that provide favorable results using the new benchmark set. While we find that VSL2B offers the most accurate and robust results overall, ESpritz-DisProt and SPOT-Disorder perform particularly well for disordered proteins. Moreover, we find that predictions for the disordered protein-binding proteins suffer low predictive quality compared to generic disordered proteins and the disordered nucleic acids-binding proteins. This can be explained by the high disorder content of the disordered protein-binding proteins, which makes it difficult for the current methods to accurately identify ordered regions in these proteins. This finding motivates the development of a new generation of methods that would target these difficult-to-predict disordered proteins. We also discuss resources that support users in collecting and identifying high-quality disorder predictions.

show abstract

Section: Surveys Of the Intrinsic Disorder Predictorsmentioning

confidence: 99%

Section: Surveys Of the Intrinsic Disorder Predictorsmentioning

confidence: 99%

Comparative Assessment of Intrinsic Disorder Predictions with a Focus on Protein and Nucleic Acid-Binding Proteins

Katuwawala

Kurgan

2020

Biomolecules

Self Cite

View full text Add to dashboard Cite

show abstract

“…Specific levels of performance on benchmark datasets, which can be gleaned from prior studies, do not guarantee that the same quality should be expected for individual proteins. A recent analysis of the protein-level performance for the prediction of the intrinsic disorder indeed shows a substantial variability of the protein-level performance [49]. Motivated by these results, we are the first to analyze differences in the quality of the predictions of RBRs across proteins and compare them to the corresponding dataset-level results.…”

Section: Introductionmentioning

confidence: 84%

Comprehensive Survey and Comparative Assessment of RNA-Binding Residue Predictions with Analysis by RNA Type

Wang

et al. 2020

IJMS

Self Cite

View full text Add to dashboard Cite

With close to 30 sequence-based predictors of RNA-binding residues (RBRs), this comparative survey aims to help with understanding and selection of the appropriate tools. We discuss past reviews on this topic, survey a comprehensive collection of predictors, and comparatively assess six representative methods. We provide a novel and well-designed benchmark dataset and we are the first to report and compare protein-level and datasets-level results, and to contextualize performance to specific types of RNAs. The methods considered here are well-cited and rely on machine learning algorithms on occasion combined with homology-based prediction. Empirical tests reveal that they provide relatively accurate predictions. Virtually all methods perform well for the proteins that interact with rRNAs, some generate accurate predictions for mRNAs, snRNA, SRP and IRES, while proteins that bind tRNAs are predicted poorly. Moreover, except for DRNApred, they confuse DNA and RNA-binding residues. None of the six methods consistently outperforms the others when tested on individual proteins. This variable and complementary protein-level performance suggests that users should not rely on applying just the single best dataset-level predictor. We recommend that future work should focus on the development of approaches that facilitate protein-level selection of accurate predictors and the consensus-based prediction of RBRs.

show abstract

“…Sequences for full-length TNIP1 were collected from UniProt release 2020_05 [40] (www.uniprot.org) (human sequence-Q15025, for a complete list of species used see supplementary materials). For review of protein disorder prediction algorithms, see [41][42][43]. For determining disorder scores of human full-length TNIP1 and TNIP1 AHD1-UBAN domains (AA417-509), web-based algorithms VL-XT, VL3, VSL2, IUPred2, MFDp2 and PONDR-FIT were used (available at http://www.pondr.com/ for VL-XT, VL3 and VSL2; https://iupred2a.elte.hu for IUPred2; http://biomine.cs.vcu.edu/servers/MFDp2/ for MFDp2; http://original.disprot.org/pondr-fit.php for PONDR-FIT).…”

Section: In Silico Analysismentioning

confidence: 99%

The Anti-Inflammatory Protein TNIP1 Is Intrinsically Disordered with Structural Flexibility Contributed by Its AHD1-UBAN Domain

2020

View full text Add to dashboard Cite

TNFAIP3 interacting protein 1 (TNIP1) interacts with numerous non-related cellular, viral, and bacterial proteins. TNIP1 is also linked with multiple chronic inflammatory disorders on the gene and protein levels, through numerous single-nucleotide polymorphisms and reduced protein amounts. Despite the importance of TNIP1 function, there is limited investigation as to how its conformation may impact its apparent multiple roles. Hub proteins like TNIP1 are often intrinsically disordered proteins. Our initial in silico assessments suggested TNIP1 is natively unstructured, featuring numerous potentials intrinsically disordered regions, including the ABIN homology domain 1-ubiquitin binding domain in ABIN proteins and NEMO (AHD1-UBAN) domain associated with its anti-inflammatory function. Using multiple biophysical approaches, we demonstrate the structural flexibility of full-length TNIP1 and the AHD1-UBAN domain. We present evidence the AHD1-UBAN domain exists primarily as a pre-molten globule with limited secondary structure in solution. Data presented here suggest the previously described coiled-coil conformation of the crystallized UBAN-only region may represent just one of possibly multiple states for the AHD1-UBAN domain in solution. These data also characterize the AHD1-UBAN domain in solution as mostly monomeric with potential to undergo oligomerization under specific environmental conditions (e.g., binding partner availability, pH-dependence). This proposed intrinsic disorder across TNIP1 and within the AHD1-UBAN region is likely to impact TNIP1 function and interaction with its multiple partners.

show abstract

Accuracy of protein-level disorder predictions

Cited by 41 publications

References 124 publications

Comparative Assessment of Intrinsic Disorder Predictions with a Focus on Protein and Nucleic Acid-Binding Proteins

Comparative Assessment of Intrinsic Disorder Predictions with a Focus on Protein and Nucleic Acid-Binding Proteins

Comprehensive Survey and Comparative Assessment of RNA-Binding Residue Predictions with Analysis by RNA Type

The Anti-Inflammatory Protein TNIP1 Is Intrinsically Disordered with Structural Flexibility Contributed by Its AHD1-UBAN Domain

Contact Info

Product

Resources

About