Cross-validation in cryo-EM–based structural modeling

Abstract

Single-particle cryo-EM is a powerful approach to determine the structure of large macromolecules and assemblies thereof in many cases at subnanometer resolution. It has become popular to refine or flexibly fit atomic models into density maps derived from cryo-EM experiments. These density maps are typically significantly lower in resolution than electron density maps obtained from X-ray diffraction experiments, such that the number of parameters that need to be determined is much larger than the number of experimental observables. Overfitting and misinterpretation of the density, thus, become a serious problem. For diffraction data, a cross-validation approach was introduced almost 20 y ago; however, no such approach has been described yet for structure refinement against cryo-EM density maps, although the overfitting problem is, because of the lower resolution, significantly larger. We present a cross-validation approach for realspace refinement against cryo-EM density maps in analogy to cross-validation typically used in crystallography. Our approach is able to detect overfitting and allows for optimizing the choice of restraints used in the refinement. The approach is shown on three protein structures with simulated data and experimental data of the rotavirus double-layer particle. Because cross-validation requires splitting the dataset into at least two independent sets, we further present an approach to quantify correlations between the structure factor sets. This analysis is also helpful for other cross-validation applications, such as refinements against diffraction data or 3D reconstructions of cryo-EM density maps.flexible fitting | real-space structure refinement S ingle-particle cryo-EM has emerged as a powerful tool to determine the structure of large biomolecular systems. The images of single particles in different orientations are recorded, which allows us to reconstruct a 3D density distribution of an average over these individual particles. These reconstructed density maps are typically in the medium-to low-resolution range of about 4-20 Å. These resolutions are usually not sufficient to directly build atomic models of macromolecules. However, in many cases, the cryo-EM density provides enough information to place individual proteins or protein domains that have been determined to higher resolution by X-ray crystallography or NMR spectroscopy or that have been built by homology modeling. Low-resolution density maps (worse than 15 Å) determine mostly the overall shape of a macromolecule, in which case the constituent individual proteins can be docked as rigid bodies into the density map (1-3), which might reveal the organization of complex protein assemblies (4-6). For the medium-resolution range between 4 and 15 Å, where more structural details are visible, a number of tools have been developed to refine or flexibly fit atomic models into density maps (7)(8)(9)(10)(11)(12)(13)(14). Some approaches use additional information during the refinement from either energy functions [e.g., MDFF (10), MDf...

Archaeal flagellin combines a bacterial type IV pilin domain with an Ig-like domain

“…The final refinement of the single-chain model was done with CNS (crystallography and NMR System), and the results from this refinement are reported in Table 1. For cross-validation, only the Fourier shells 130.0-4.3 Å were used for the refinement, and the shells 4.3-4.0 Å were used for calculating the C free , R free , and Fourier shell correlation [FSC; FSC avg (free)] values as described (58). To include subunit interactions, a partial filament structure was built for 21 subunits.…”

Validation methods for low-resolution fitting of atomic structures to electron microscopy data

“…In the current context of fitting atomic structures into cryo-EM reconstructions, two conceptually different approaches have been proposed [32,33]. The first approach [32] relies on splitting the data according to frequency ranges where low-resolution frequency data with significant signal-to-noise ratio is used as a training set for model building and a shell of high-frequency data is used as a test set.…”

“…The first approach [32] relies on splitting the data according to frequency ranges where low-resolution frequency data with significant signal-to-noise ratio is used as a training set for model building and a shell of high-frequency data is used as a test set. As expected, significant correlations between the training and test sets can be detected.…”

Asymmetric perturbations of signalling oligomers

“…It is important to note though that averaging and symmetry constraints used in EM often eliminate the chance of detecting oligomeric and small local asymmetries. Complementary information (XRC) and cross-validation can improve the evaluation of low-resolution cryo-EM data and prevent overfitting (Abrahams and Ban, 2003;Falkner and Schröder, 2013).…”