Single particle cryo-EM requires full automation to allow high-throughput structure determination. Although software packages exist where parts of the cryo-EM pipeline are automated, a complete solution that offers reliable on-the-fly processing, resulting in high-resolution structures, does not exist. Here we present TranSPHIRE: A software package for fully-automated processing of cryo-EM datasets during data acquisition. TranSPHIRE transfers data from the microscope, automatically applies the common pre-processing steps, picks particles, performs 2D clustering, and 3D refinement parallel to image recording. Importantly, TranSPHIRE introduces a machine learning-based feedback loop to re-train its picking model to adapt to any given data set live during processing. This elegant approach enables TranSPHIRE to process data more effectively, producing high-quality particle stacks. TranSPHIRE collects and displays all metrics and microscope settings to allow users to quickly evaluate data during acquisition. TranSPHIRE can run on a single work station and also includes the automated processing of filaments.

9Single particle electron cryomicroscopy (cryo-EM) requires full automation to allow 10 high-throughput structure determination which is especially important for drug 11 discovery research. Although several software packages exist where parts of the 12 cryo-EM pipeline are automated, a complete solution that offers reliable, quality-13 optimized on-the-fly processing, resulting in a high-resolution three-dimensional 14 reconstruction does not exist. Here we present TranSPHIRE: A software package for 15 fully automated processing of cryo-EM data sets during data acquisition. TranSPHIRE 16 transfers data from the microscope, automatically applies the common pre-17 processing steps, picks particles, performs 2D clustering, and 3D refinement parallel 18 to image recording. Importantly, TranSPHIRE introduces a machine learning-based 19 feedback loop to re-train its internally used picking model to adapt to any given data 20 set live during processing. This elegant approach enables TranSPHIRE to process data 21 more effectively, producing high-quality particle stacks. TranSPHIRE collects, and 22 displays all microscope settings and metrics generated by its individual tools, in order 23 to allow users to quickly evaluate data during acquisition. TranSPHIRE can run on a 24 single work station and also includes the automated processing of filaments. 25 26Although several software packages partially address these issues 4-10 , a 46 complete solution that offers reliable, quality-optimized and flexible on-the-fly 47 processing during data acquisition resulting in a high-resolution 3D reconstruction 48does not yet exist. CryoFLARE 4 , for example, performs live analysis and processing 49 parallel to data acquisition, but only to the level of 2D classification and lacks the 50 ability to perform ab initio 3D reconstructions or high resolution refinements. 51Similarly, Focus 8 , Appion 10 , and Warp 6 do not produce 2D class averages and 3D 52 3 reconstructions. The latter two are less flexible than other offerings by being 53 restricted to either collecting data with Leginon 11 in case of Appion or exclusive 54 compatibility with Windows and Warp-native tools. All three software packages 55 concentrate on data acquisition and associated parameters but not on the 56 optimization of data processing which is an important prerequisite for automated 57 structure determination. The non-interactive data pre-processing in Relion-3 5 offers, 58 similar to Focus 8 , some flexibility in terms of tool integration, but hinders the 59 implementation of more complicated cryo-EM processing by making advanced 60 parameters only accessible via manual scripting, rather than its GUI. Both Relionit 5 61 and Scipion 7 share the same accessibility issue of quality metrics, where no values 62 are automatically plotted and updated during processing. Instead, the user has to 63 step in and trigger the compilation of a log-file that contains a mix of metrics for all 64 processed data; any specific values of an individual micrograph have to be found 65 ...

A widely used approach to analyze single particles in electron microscopy data is 2D classification. This process is very computationally expensive, especially when large data sets are analyzed. In this paper we present GPU ISAC, a newly developed, GPU-accelerated version of the established Iterative Stable Alignment and Clustering (ISAC) algorithm for 2D images and generating class averages. While the previously existing implementation of ISAC relied on a computer cluster, GPU ISAC enables users to produce high quality 2D class averages from large-scale data sets on a single desktop machine equipped with affordable, consumer-grade GPUs such as Nvidia GeForce GTX 1080 TI cards. With only two such cards GPU ISAC matches the performance of twelve high end cluster nodes and, using high performance GPUs, is able to produce class averages from a million particles in between six to thirteen hours, depending on data set quality and box size. We also show GPU ISAC to scale linearly in all input dimensions, and thereby capable of scaling well with the increasing data load demand of future data sets. Further user experience improvements integrate GPU ISAC seamlessly into the existing SPHIRE GUI, as well as the TranSPHIRE on-the-fly processing pipeline. It is open source and can be downloaded at https://gitlab.gwdg.de/mpi-dortmund/sphire/cuISAC/