the quality of super-resolution images obtained by singlemolecule localization microscopy (smlm) depends largely on the software used to detect and accurately localize point sources. in this work, we focus on the computational aspects of super-resolution microscopy and present a comprehensive evaluation of localization software packages. our philosophy is to evaluate each package as a whole, thus maintaining the integrity of the software. We prepared synthetic data that represent three-dimensional structures modeled after biological components, taking excitation parameters, noise sources, point-spread functions and pixelation into account. We then asked developers to run their software on our data; most responded favorably, allowing us to present a broad picture of the methods available. We evaluated their results using quantitative and user-interpretable criteria: detection rate, accuracy, quality of image reconstruction, resolution, software usability and computational resources. these metrics reflect the various tradeoffs of smlm software packages and help users to choose the software that fits their needs.We have conducted a large-scale comparative study of software packages developed in the context of SMLM, including recently developed algorithms. We designed realistic data that are generic and cover a broad range of experimental conditions and compared the software packages using a multiple-criterion quantitative assessment that is based on a known ground truth.Our study is based on the active participation of developers of SMLM software. More than 30 groups have participated so far, and the study is still under way. We provide participants access to our benchmark data as an ongoing public challenge. Participants run their own software on our data and report their list of localized particles for evaluation. The results of the challenge are accessible online and updated regularly.SMLM was demonstrated in 2006, independently by three research groups 1-3 , and has enabled subsequent breakthroughs in diverse fields 4,5 . SMLM can resolve biological structures at the nanometer scale (typically 20 nm lateral resolution), circumventing Abbe's diffraction limit. At the cost of a relatively simple setup 6,7 , it has opened exciting new opportunities in life science research 8,9 .The underlying principle of SMLM is the sequential imaging of sparse subsets of fluorophores distributed over thousands of frames, to populate a high-density map of fluorophore positions. Such large data sets require automated image-analysis algorithms to detect and precisely infer the position of individual fluorophore, taking advantage of their separation in space and time.The acquired data cannot be visualized directly; further computerized image-reconstruction methods are required. These typically comprise four steps: preprocessing, detection, localization and rendering. Preprocessing reduces the effects of the background and noise; detection identifies potential molecule candidates in each frame; localization performs a subpixel refine...