We present deep-learning-enabled super-resolution across different fluorescence microscopy modalities. This data-driven approach does not require numerical modeling of the imaging process or the estimation of a point-spread-function, and is based on training a generative adversarial network (GAN) to transform diffraction-limited input images into super-resolved ones. Using this Reprints and permissions information is available at www.nature.com/reprints.

Early identification of pathogenic bacteria in food, water, and bodily fluids is very important and yet challenging, owing to sample complexities and large sample volumes that need to be rapidly screened. Existing screening methods based on plate counting or molecular analysis present various tradeoffs with regard to the detection time, accuracy/sensitivity, cost, and sample preparation complexity. Here, we present a computational live bacteria detection system that periodically captures coherent microscopy images of bacterial growth inside a 60-mm-diameter agar plate and analyses these time-lapsed holograms using deep neural networks for the rapid detection of bacterial growth and the classification of the corresponding species. The performance of our system was demonstrated by the rapid detection of Escherichia coli and total coliform bacteria (i.e., Klebsiella aerogenes and Klebsiella pneumoniae subsp. pneumoniae) in water samples, shortening the detection time by >12 h compared to the Environmental Protection Agency (EPA)-approved methods. Using the preincubation of samples in growth media, our system achieved a limit of detection (LOD) of~1 colony forming unit (CFU)/L in ≤9 h of total test time. This platform is highly cost-effective (~$0.6/test) and has high-throughput with a scanning speed of 24 cm 2 /min over the entire plate surface, making it highly suitable for integration with the existing methods currently used for bacteria detection on agar plates. Powered by deep learning, this automated and cost-effective live bacteria detection platform can be transformative for a wide range of applications in microbiology by significantly reducing the detection time and automating the identification of colonies without labelling or the need for an expert.

We present a deep learning-based method for achieving super-resolution in fluorescence microscopy. This data-driven approach does not require any numerical models of the imaging process or the estimation of a point spread function, and is solely based on training a generative adversarial network, which statistically learns to transform low resolution input images into super-resolved ones. Using this method, we super-resolve wide-field images acquired with low numerical aperture objective lenses, matching the resolution that is acquired using high numerical aperture objectives. We also demonstrate that diffraction-limited confocal microscopy images can be transformed by the same framework into super-resolved fluorescence images, matching the image resolution acquired with a stimulated emission depletion (STED) microscope. The deep network rapidly outputs these super-resolution images, without any iterations or parameter search, and even works for types of samples that it was not trained for.Computational super-resolution microscopy techniques in general make use of a priori knowledge about the sample and/or the image formation process to enhance the resolution of an acquired image. At the heart of the existing super-resolution methods 1-3 , numerical models are utilized to simulate the imaging process, including, for example, an estimation of the point spread function (PSF) of the imaging system, its spatial sampling rate and/or sensor-specific noise patterns. Fluorescence imaging process is in general more challenging to model and take into account e.g., spatially-varying optical aberrations, the chemical environment of the labeled sample and the optical properties of the specific mounting media and the fluorophores that are used 4-7 . This image modeling related challenge, in turn, leads to formulation of forward models with different simplifying assumptions. In general, more accurate models yield higher quality results, often with a trade-off of exhaustive parameter search and computational cost.Here we present a deep learning-based framework to achieve super-resolution in fluorescence microscopy without the need for making any assumptions on or precise modeling of the image formation process. Instead, we train a deep neural network using a Generative Adversarial Network (GAN) 8 model to transform an acquired low-resolution image into a high-resolution one. Once the deep network is trained (see the Methods section), it remains fixed and can be used to rapidly output batches of high resolution images, in e.g., 0.4 sec for an image size of 1024×1024 pixels using a single Graphics Processing Unit (GPU). The network inference is noniterative and does not require a manual parameter search to optimize its algorithmic performance.The deep network can also be generalized to different types of samples that were not part of the training process.We demonstrate the success of this deep learning-based approach by super-resolving the raw images captured by a widefield fluorescence microscope and a confocal microscope. In the...

No abstract

No abstract