Evaluation system and web infrastructure for the second cryo-EM model challenge

Abstract

An evaluation system and a web infrastructure were developed for the second cryo-EM model challenge. The evaluation system includes tools to validate stereo-chemical plausibility of submitted models, check their fit to the corresponding density maps, estimate their overall and per-residue accuracy, and assess their similarity to reference cryo-EM or X-ray structures as well as other models submitted in this challenge. The web infrastructure provides a convenient interface for analyzing models at different levels of detail. It includes interactively sortable tables of evaluation scores for different subsets of models and different sublevels of structure organization, and a suite of visualization tools facilitating model analysis. The results are publicly accessible at http://model-compare.emdatabank.org.

Cryo‐electron microscopy targets in CASP13: Overview and evaluation of results

“…CASP infrastructure for running the evaluation, reporting scores, and visualizing evaluation results for cryo‐EM targets (http://predictioncenter.org/casp13/cryoem_results.cgi) is based on the prototype of the evaluation infrastructure developed for the cryo‐EM model challenge …”

Distribution of evaluation scores for the models submitted to the second cryo-EM model challenge

“…8 of Ref. [3] for all submitted models (optimization and ab initio models pulled together). Left set of boxplots shows scores from multimeric evaluations, right set – from monomeric ones.…”

“…142 protein structure models were submitted on eight modeling targets of the second Cryo-EM model challenge (2015–2016) [2] . A computational system was developed to estimate the accuracy of the models [3] . Each model was evaluated using a suite of 15 software packages and, as a result, 54 accuracy scores were generated per model.…”

“…All scores are presented in tables and graphs of the dedicated web infrastructure ( http://model-compare.emdatabank.org ). Some scores were analyzed in the accompanying paper [3] . This article provides results of the descriptive statistical analysis of the complete set of evaluation scores.…”

Comparing cryo-EM structures

“…Model assessment was supported by the creation of an analysis pipeline built on a prototype of the Critical Assessment of protein Structure Prediction (CASP) evaluation system (Kryshtafovych et al, 2018, this issue). Multiple global and local measures for model geometry, reference model similarity, and fit-to-map were compiled using fifteen different software tools.…”