Assessment of the assessment: Evaluation of the model quality estimates in CASP10

Kryshtafovych, Andriy; Barbato, Alessandro; Fidelis, Krzysztof; Monastyrskyy, Bohdan; Schwede, Torsten; Tramontano, Anna

doi:10.1002/prot.24347

Cited by 129 publications

(168 citation statements)

References 32 publications

(45 reference statements)

Supporting

Mentioning

164

Contrasting

Unclassified

Order By: Relevance

“…Every second year the performance of the best quality estimation methods are evaluated in CASP [5]. The purpose of this category is to test the current state-of-the-art methods for their ability to give an absolute estimate of the quality of an ensemble of decoys (near-native configurations).…”

Section: Model Quality Assessment and Model Refinementmentioning

confidence: 99%

“…A critical component of the CASP experiment is the assessment of the predictions that are submitted as putative models for the target proteins considered. This assessment is performed by the CASP assessors but also by the CASP community, with considerable enthusiasm, as observed in CASP10 [95]. The procedure for assessing the predictions in CASP10 differed from that of previous CASPs.…”

Section: Training and Test Setsmentioning

confidence: 99%

“…f The Stage 1 and Stage 2 decoy sets used in the CASP10 quality assessment category, available from the CASP web site http://predictioncenter.org. For details on how these sets are prepared, see [95]. g All high and low resolution targets (TSA TM-score > 0.5)/(TSA TM-score < 0.5) are listed in the files S TSAh and S TSAl respectively found in the supporting information.…”

Section: Assessing the Quality Of Decoy Selection: R-scorementioning

confidence: 99%

“…Knowledgebased potentials are trained with the purpose of improving the quality of near-native structures. This means either to rank a set of near-native protein structures according to how native-like they are (model quality assessment [5]) or to refine the quality of near-native protein structures (protein structure refinement [6]). Most knowledgebased potentials are stochastic but in this study a knowledge-based potential is a model of the energy landscape of proteins that is trained on a set of near-native structures called decoys.…”

Section: Introduction 1 Introductionmentioning

confidence: 99%

“…These energy functions are trained and tested on sets extracted from the high resolution decoy dataset Titan-HRD [47], as well as on well known decoy datasets from DecoysRUs [93] and Rosetta [94]. We have also analyzed the performance of our potentials on the server generated Stage 1 and Stage 2 decoy sets from CASP 10 [95].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Protein structure refinement by optimization

Carlsen

Røgen

2015

Proteins

View full text Add to dashboard Cite

SummaryProteins are the main active elements of life whose chemical activities regulate cellular activities. A protein is characterized by having a sequence of amino acids and a three dimensional structure. The three-dimensional structure has only been determined experimentally for 50000 of the seven million sequences that are known. Determining the protein structure from its sequence of amino acids is therefore a major problem in computational structural biology and is referred to as the protein folding problem. The folding problem is solved using de novo methods or comparative methods depending on whether the three-dimensional structure of a homologous sequence is known. Whether or not a protein model can be used for industrial purposes depends on the quality of the predicted structure. A model can be used to design a drug when the quality is high.The overall goal of this project is to assess and improve the quality of a predicted structure. The starting point of this work is a technique called metric training where a knowledge-based protein potential, for a fixed set of native protein structures and a set of deformed decoys for each native structure, is designed to have native-decoy energy gabs that correlates maximally to a native-decoy distance. The main contribution of this thesis is methods developed for analyzing the performance of metrically trained knowledge-based potentials and for optimizing their performance while making them less dependent on the decoy set used to define them. We focus on using the gradient and the Hessian in the analysis and present a novel smooth solvation potential but otherwise the studied potential is kept close to standard coarse grained potentials.We analyze the importance of the choice of metric both when used in metric training and when used in the evaluation of the performance of the resulting potential and find a significant improvement by using a metric based on intrinsic geometry. It is well-known that energy minimization of a potential that is efficient in ordering a fixed set of decoys need not bring the decoys closer to the native state. The next part of the work is focused on improving the convergence of decoy structures and we present a method that significantly improves the results of shorter energy minimizations of a metrically trained potential and discuss its limitations. In an ideal potential all nearnative decoys will converge toward the native structure being at-least a local minimum of the potential. To address how far the current functional form of the potential is from an ideal potential we present two methods for finding the optimal metrically trained potential that simultaneous has a number of native structures as a local minimum. Our results generally indicate that a more fine-grained potential is needed to meet desired model accuracies but even with our coarse-grained model we obtain good results and there is an unexplored possibility to combine it with comparative modeling.To allow fast energy minimization in Matlab a new set of more sparse formulas...

show abstract

Section: Model Quality Assessment and Model Refinementmentioning

confidence: 99%

Section: Training and Test Setsmentioning

confidence: 99%

Section: Assessing the Quality Of Decoy Selection: R-scorementioning

confidence: 99%

Section: Introduction 1 Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Protein structure refinement by optimization

Carlsen

Røgen

2015

Proteins

View full text Add to dashboard Cite

show abstract

Use of a structural alphabet to find compatible folds for amino acid sequences

et al. 2014

View full text Add to dashboard Cite

The structural annotation of proteins with no detectable homologs of known 3D structure identified using sequence-search methods is a major challenge today. We propose an original method that computes the conditional probabilities for the amino-acid sequence of a protein to fit to known protein 3D structures using a structural alphabet, known as "Protein Blocks" (PBs). PBs constitute a library of 16 local structural prototypes that approximate every part of protein backbone structures. It is used to encode 3D protein structures into 1D PB sequences and to capture sequence to structure relationships. Our method relies on amino acid occurrence matrices, one for each PB, to score global and local threading of query amino acid sequences to protein folds encoded into PB sequences. It does not use any information from residue contacts or sequencesearch methods or explicit incorporation of hydrophobic effect. The performance of the method was assessed with independent test datasets derived from SCOP 1.75A. With a Z-score cutoff that achieved 95% specificity (i.e., less than 5% false positives), global and local threading showed sensitivity of 64.1% and 34.2%, respectively. We further tested its performance on 57 difficult CASP10 targets that had no known homologs in PDB: 38 compatible templates were identified by our Additional Supporting Information may be found in the online version of this article. approach and 66% of these hits yielded correctly predicted structures. This method scales-up well and offers promising perspectives for structural annotations at genomic level. It has been implemented in the form of a web-server that is freely available at http://www.bo-protscience.fr/forsa.

show abstract

Using physical features of protein core packing to distinguish real proteins from decoys

et al. 2020

View full text Add to dashboard Cite

The ability to consistently distinguish real protein structures from computationally generated model decoys is not yet a solved problem. One route to distinguish real protein structures from decoys is to delineate the important physical features that specify a real protein. For example, it has long been appreciated that the hydrophobic cores of proteins contribute significantly to their stability. We used two sources to obtain datasets of decoys to compare with real protein structures: submissions to the biennial Critical Assessment of protein Structure Prediction competition, in which researchers attempt to predict the structure of a protein only knowing its amino acid sequence, and also decoys generated by 3DRobot, which have user‐specified global root‐mean‐squared deviations from experimentally determined structures. Our analysis revealed that both sets of decoys possess cores that do not recapitulate the key features that define real protein cores. In particular, the model structures appear more densely packed (because of energetically unfavorable atomic overlaps), contain too few residues in the core, and have improper distributions of hydrophobic residues throughout the structure. Based on these observations, we developed a feed‐forward neural network, which incorporates key physical features of protein cores, to predict how well a computational model recapitulates the real protein structure without knowledge of the structure of the target sequence. By identifying the important features of protein structure, our method is able to rank decoy structures with similar accuracy to that obtained by state‐of‐the‐art methods that incorporate many additional features. The small number of physical features makes our model interpretable, emphasizing the importance of protein packing and hydrophobicity in protein structure prediction.

show abstract

Assessment of the assessment: Evaluation of the model quality estimates in CASP10

Cited by 129 publications

References 32 publications

Protein structure refinement by optimization

Protein structure refinement by optimization

Use of a structural alphabet to find compatible folds for amino acid sequences

Using physical features of protein core packing to distinguish real proteins from decoys

Contact Info

Product

Resources

About