To split or not to split: CASP15 targets and their processing into tertiary structure evaluation units

Self Cite

Computing protein structure from amino acid sequence information has been a long‐standing grand challenge. Critical assessment of structure prediction (CASP) conducts community experiments aimed at advancing solutions to this and related problems. Experiments are conducted every 2 years. The 2020 experiment (CASP14) saw major progress, with the second generation of deep learning methods delivering accuracy comparable with experiment for many single proteins. There is an expectation that these methods will have much wider application in computational structural biology. Here we summarize results from the most recent experiment, CASP15, in 2022, with an emphasis on new deep learning‐driven progress. Other papers in this special issue of proteins provide more detailed analysis. For single protein structures, the AlphaFold2 deep learning method is still superior to other approaches, but there are two points of note. First, although AlphaFold2 was the core of all the most successful methods, there was a wide variety of implementation and combination with other methods. Second, using the standard AlphaFold2 protocol and default parameters only produces the highest quality result for about two thirds of the targets, and more extensive sampling is required for the others. The major advance in this CASP is the enormous increase in the accuracy of computed protein complexes, achieved by the use of deep learning methods, although overall these do not fully match the performance for single proteins. Here too, AlphaFold2 based method perform best, and again more extensive sampling than the defaults is often required. Also of note are the encouraging early results on the use of deep learning to compute ensembles of macromolecular structures. Critically for the usability of computed structures, for both single proteins and protein complexes, deep learning derived estimates of both local and global accuracy are of high quality, however the estimates in interface regions are slightly less reliable. CASP15 also included computation of RNA structures for the first time. Here, the classical approaches produced better agreement with experiment than the new deep learning ones, and accuracy is limited. Also, for the first time, CASP included the computation of protein–ligand complexes, an area of special interest for drug design. Here too, classical methods were still superior to deep learning ones. Many new approaches were discussed at the CASP conference, and it is clear methods will continue to advance.

Section: Resultsmentioning

confidence: 99%

Section: Domain-domain Interactionsmentioning

confidence: 99%

Section: Domain-domain Interactionsmentioning

confidence: 99%

See 1 more Smart Citation

Critical assessment of methods of protein structure prediction (CASP)—Round XV

Kryshtafovych,

Schwede,

Topf

et al. 2023

Self Cite

“…Rigden for details of the CASP15 EUs definition). 87 The tertiary structure assessment concentrates on the question of how well structures of separate domains are reproduced, while the interplay between domains is left outside the scope of that assessment and is the subject of this study. All in all, 22 single-sequence CASP15 targets (monomers and subunits of multimers) were split into multiple EUs (called here for simplicity "domains").…”

Section: From Inter-monomer To Interdomain Predictionsmentioning

confidence: 99%

The impact of AI‐based modeling on the accuracy of protein assembly prediction: Insights from CASP15

Ozden,

Kryshtafovych,

Karaca

2023

Self Cite

In CASP15, 87 predictors submitted around 11 000 models on 41 assembly targets. The community demonstrated exceptional performance in overall fold and interface contact predictions, achieving an impressive success rate of 90% (compared to 31% in CASP14). This remarkable accomplishment is largely due to the incorporation of DeepMind's AF2‐Multimer approach into custom‐built prediction pipelines. To evaluate the added value of participating methods, we compared the community models to the baseline AF2‐Multimer predictor. In over 1/3 of cases, the community models were superior to the baseline predictor. The main reasons for this improved performance were the use of custom‐built multiple sequence alignments, optimized AF2‐Multimer sampling, and the manual assembly of AF2‐Multimer‐built subcomplexes. The best three groups, in order, are Zheng, Venclovas, and Wallner. Zheng and Venclovas reached a 73.2% success rate over all (41) cases, while Wallner attained 69.4% success rate over 36 cases. Nonetheless, challenges remain in predicting structures with weak evolutionary signals, such as nanobody–antigen, antibody–antigen, and viral complexes. Expectedly, modeling large complexes also remains challenging due to their high memory compute demands. In addition to the assembly category, we assessed the accuracy of modeling interdomain interfaces in the tertiary structure prediction targets. Models on seven targets featuring 17 unique interfaces were analyzed. Best predictors achieved a 76.5% success rate, with the UM‐TBM group being the leader. In the interdomain category, we observed that the predictors faced challenges, as in the case of the assembly category, when the evolutionary signal for a given domain pair was weak or the structure was large. Overall, CASP15 witnessed unprecedented improvement in interface modeling, reflecting the AI revolution seen in CASP14.

“…9 A group ranking was generated as follows using the complete chain submissions submitted by groups instead of the CASP-defined Evaluation Units (EUs). 13 Predictors may submit five models for each target. To reduce the computational time required for the docking process, only the first submitted model for each target per group was considered.…”

Section: Ranking Of Individual Protein Chains Based On Rigid Cryo-em ...mentioning

confidence: 99%

CASP15 cryo‐EM protein and RNA targets: Refinement and analysis using experimental maps

Mulvaney,

Kretsch,

Elliott

et al. 2023

Self Cite

CASP assessments primarily rely on comparing predicted coordinates with experimental reference structures. However, experimental structures by their nature are only models themselves—their construction involves a certain degree of subjectivity in interpreting density maps and translating them to atomic coordinates. Here, we directly utilized density maps to evaluate the predictions by employing a method for ranking the quality of protein chain predictions based on their fit into the experimental density. The fit‐based ranking was found to correlate well with the CASP assessment scores. Overall, the evaluation against the density map indicated that the models are of high accuracy, and occasionally even better than the reference structure in some regions of the model. Local assessment of predicted side chains in a 1.52 Å resolution map showed that side‐chains are sometimes poorly positioned. Additionally, the top 118 predictions associated with 9 protein target reference structures were selected for automated refinement, in addition to the top 40 predictions for 11 RNA targets. For both proteins and RNA, the refinement of CASP15 predictions resulted in structures that are close to the reference target structure. This refinement was successful despite large conformational changes often being required, showing that predictions from CASP‐assessed methods could serve as a good starting point for building atomic models in cryo‐EM maps for both proteins and RNA. Loop modeling continued to pose a challenge for predictors, and together with the lack of consensus amongst models in these regions suggests that modeling, in combination with model‐fit to the density, holds the potential for identifying more flexible regions within the structure.