Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software

Abstract

With the widespread uptake of 2D and 3D single molecule localization microscopy, a large set of different data analysis packages have been developed to generate super-resolution images. In a large community effort we designed a competition to extensively characterise and rank the performance of 2D and 3D single molecule localization microscopy software packages. We generated realistic simulated datasets for popular imaging modalities-2D, astigmatic 3D, biplane 3D, and double helix 3D-and evaluated 36 participant packages against these data. This provides the first broad assessment of 3D single molecule localization microscopy software and provides a holistic view of how the latest 2D and 3D single molecule localization software perform in realistic conditions. This resource allows researchers to identify optimal analytical software for their experiments, allows 3D SMLM software developers to benchmark new software against current state of the art, and provides insight into the current limits of the field. RESULTS Competition design We established a broad committee from the SMLM community, including experimentalists and software developers, to define the scope of the challenge, ensure realism of the datasets and define analysis metrics. We opened this discussion to all interested parties in an online discussion forum 17. In 2016, we ran a first round of the 3D SMLM competition with explicit submission deadlines, culminating in a special session at the 6th annual Single Molecule Localization Microscopy Symposium (SMLMS 2016). Since then, the challenge has been opened to continuously accept new entries. Thirtysix software packages have been entered in the competition thus far, including four packages used in commercial software (Table S1, Supplementary Note 1). Participation in the competition actually led at least eight teams to modify their software to support additional 3D SMLM modalities, showing how competition can foster microscopy software development. Realistic 3D simulations Testing super-resolution software on experimental data lacks the ground truth information required for rigorous quantification of software performance. Therefore, realistic simulated datasets are required. A critical challenge to in simulating 3D SMLM data was accurate modeling of the

About samples, giving examples: Optimized Single Molecule Localization Microscopy

“…Huang et al, 2008). Due to the enlarged PSF and the added dimension, algorithm for 3D SMLM usually have optimal performance at densities below those attainable in 2D SMLM (Sage et al, 2019), so this has to be taken into account during acquisition. 3D STORM images can be represented as Z color-coded images or transverse XZ/YZ sections showing the 3D organization of structures such as microtubules, clathrin-coated pits and actin filaments (Fig.…”

“…For densely labeled samples, specific algorithms are available that are able to fit overlapping peaks (Holden et al, 2011;Mailfert et al, 2018;Sage et al, 2019). Detection threshold (to decide what will be fitted) and rejection parameters (to avoid fitting peaks that are not single molecule blinking events) are important and should be adjusted to the acquisition conditions.…”

“…SMLM localization algorithms are now quite mature, with several software showing good performance. Rigorous evaluations of performance on standardized benchmark tests have been published for a number of software options both for 2D and 3D-SMLM, providing an unbiased overview of the available options (Sage et al, 2019;2015). We will mention a few that we have tested extensively and found to give qualitatively satisfactory results on our data.…”

FOCAL3D: A 3-dimensional clustering package for single-molecule localization microscopy

“…We now apply our clustering algorithm to 3-dimensional astigmatic dSTORM images of the nuclear pore complex (NPC) ( [14,15,16]). The images were acquired in wild-type U-2 human osteosarcoma (OS) cells expressing Nup107-SNAP, which was fluorescently labeled with the organic dye Alexa-647 and induced to photoswitch by modifying the imaging bu↵er.…”

Verifying molecular clusters by 2-color localization microscopy and significance testing

“…To make the method immediately applicable, we provide a plugin for ImageJ (see Supporting Material). 26 The experimental basis is a chromatically corrected two-color SMLM data-set analyzed by standard single 27 molecule localization tools (Sage et al, 2019). 28…”