An analysis and evaluation of the WeFold collaborative for protein structure prediction and its pipelines in CASP11 and CASP12

Abstract

Every two years groups worldwide participate in the Critical Assessment of Protein Structure Prediction (CASP) experiment to blindly test the strengths and weaknesses of their computational methods. CASP has significantly advanced the field but many hurdles still remain, which may require new ideas and collaborations. In 2012 a web-based effort called WeFold, was initiated to promote collaboration within the CASP community and attract researchers from other fields to contribute new ideas to CASP. Members of the WeFold coopetition (cooperation and competition) participated in CASP as individual teams, but also shared components of their methods to create hybrid pipelines and actively contributed to this effort. We assert that the scale and diversity of integrative prediction pipelines could not have been achieved by any individual lab or even by any collaboration among a few partners. The models contributed by the participating groups and generated by the pipelines are publicly available at the WeFold website providing a wealth of data that remains to be tapped. Here, we analyze the results of the 2014 and 2016 pipelines showing improvements according to the CASP assessment as well as areas that require further adjustments and research.

Loss-functions matter, on optimizing score functions for the estimation of protein models accuracy

“…Aiming at both absolute and relative accuracy prediction, earlier versions of MESHI-Score used loss-functions that combined a penalty on per-target mean error and a reward for higher Pearson correlation. The utility of the resulted score functions has been demonstrated in CASP11 and CASP12 experiments (Elofsson et al, 2017;Keasar et al, 2018). The current study relaxes the original goal, and seek specialized score functions for either absolute or relative accuracy.…”

“…We are not aware of later studies that followed this interesting approach. MESHI-Score (Elofsson et al, 2017;Keasar et al, 2018;Mirzaei et al, 2016) is a supervised learning method for EMA, that trains its statistical model by Monte-Carlo Simulated Annealing (MCSA) optimization (Brooks and Morgan, 1995;Kirkpatrick, 1984;Metropolis and Ulam, 1949). MCSA is a very flexible computational approach, and is specifically permissive towards the nature of the loss-function that guides optimization.…”

Assessment of protein model structure accuracy estimation in CASP13: Challenges in the era of deep learning

“…However, such meta‐servers do not have scientific or practical value because they rely heavily on the models generated only in the CASP setting. In several latest rounds of CASP, an interesting community‐wide collaboration emerged (WeFold), where methods from different prediction groups were combined for model generation, EMA, and structure refinement . However, more intricate combinations of EMA with other structure prediction components beyond simple scoring of pregenerated models have yet to come.…”

Alternative Hydrophobic Core in Proteins—The Effect of Specific Synergy

“…However, the effects of this project are unsatisfactory [21,22]. Creating a joint team consisting of top groups did not give the expected effect [23,24]. The WeFold project [23,24] shows that the traditional division into homology-based methods (generating structure based on knowledge of evolutionally related protein structures) and ab initio (model of the folding mechanism without reference to other known proteins) has exhausted its possibilities.…”