Deep learning in single-molecule microscopy: fundamentals, caveats, and recent developments [Invited]

Möckl, Leonhard; Roy, Anish R.; Moerner, W. E.

doi:10.1364/boe.386361

Cited by 78 publications

(68 citation statements)

References 84 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, deep learning has been successfully implemented for detection and localization of PSFs, 189 − 191 for phase retrieval and background correction, 192 − 195 for phase mask design of optimized PSFs, 157 , 196 and for compressed sensing, 197 , 198 to just name a few applications, and work in this area has been recently reviewed. 199 …”

Section: Perspectives On Methods and Applicationsmentioning

confidence: 99%

Super-resolution Microscopy with Single Molecules in Biology and Beyond–Essentials, Current Trends, and Future Challenges

2020

Self Cite

View full text Add to dashboard Cite

Single-molecule super-resolution microscopy has developed from a specialized technique into one of the most versatile and powerful imaging methods of the nanoscale over the past two decades. In this perspective, we provide a brief overview of the historical development of the field, the fundamental concepts, the methodology required to obtain maximum quantitative information, and the current state of the art. Then, we will discuss emerging perspectives and areas where innovation and further improvement are needed. Despite the tremendous progress, the full potential of single-molecule super-resolution microscopy is yet to be realized, which will be enabled by the research ahead of us.

show abstract

Section: Perspectives On Methods and Applicationsmentioning

confidence: 99%

Super-resolution Microscopy with Single Molecules in Biology and Beyond–Essentials, Current Trends, and Future Challenges

2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…These machine learning algorithms could be transformative for the analysis of complex single-molecule data, because they could be used to quantify complex multi-state data. In fact, neural network techniques have already been applied to image processing in super-resolution microscopy, [167] fluorescence-based detection, [168] and also for signal analysis in nanopore sensors, [169,170] ion channel patch-clamps, [171] and DNA sequencers. [169,172] In the latter example, a neural network was applied to replace the hidden Markov model in a commercial nanopore-based DNA sequencer.…”

Section: Improving Data Analysismentioning

confidence: 99%

Plasmonic Assemblies for Real‐Time Single‐Molecule Biosensing

Armstrong

Horáček

Zijlstra

2020

Small

View full text Add to dashboard Cite

and bound states in the continuum, [27,28] and have been used in applications like biosensing [29] and color printing. [30] Plasmonic assemblies have proven particularly powerful in the field of biosensing. The presence of a biomarker results in, e.g., the (dis-)assembly of solution-phase plasmonic particles, or to changes in the particle-spacing in an assembly. Both result in strong changes in the far-field optical response of the plasmonic assemblies that is most often monitored using spectroscopy in a UV-vis spectrometer. In solution-based sensors, clustering of metal nanoparticles can be induced by target molecules, shifting the optical extinction spectrum of the nanoparticle solution (Figure 1d). [31] Such clustering assays based on particle (dis-) assembly have resulted in colorimetric assays for a range of analytes including proteins, DNA/RNA sequences, heavy ions, and toxic compounds. [31,32] Single-molecule biosensors, on the other hand, detect analytes by probing the optical response of a single nanoparticle. An important advantage of single-molecule sensing is the ability to construct histograms of sensor properties at the single-molecule level, e.g., the induced plasmon shift and statistical parameters like the waiting-time between events. Single-molecule biosensing using plasmonic sensors has been demonstrated in 2012 by Figure 1. Reported applications of nanoparticle assemblies. a) Left: Calculated SERS enhancement factors for a sphere dimer with a 2 nm gap spacing. Right: A Raman spectra of methylene blue dispersed on a monolayer (black) and bilayer (red) of gold nanoparticles. Adapted with permission. [11] Copyright 2019, American Chemical Society. (https://pubs.acs.org/doi/10.1021/acsnano.9b04224) Further permissions related to this material should be directed to the ACS. b) Non-linear emission spectrum of noncentrosymmetric patterned gold nanorods. The third-harmonic generation (THG) peak is in green and the second-harmonic generation (SHG) is in orange. Top inset-scheme of the assembly's THG (green) and SHG (orange) stemming from the red excitation wavelength (ω). Bottom inset-electron microscopy image of the nanoassembly (scale bar 200 nm). Adapted with permission. [16] Copyright 2019, American Chemical Society. c) Examples of metamaterials comprised of plasmonic assemblies (scale bars are 500 nm). Reproduced with permission. [25] Copyright 2014, Springer Nature. d) Left: A cluster assay biosensor in which nanoparticle assembly is induced by a target molecule. Right: An example UV-vis spectrum of the gold nanoparticle monomers (red) and subsequent assemblies upon addition of target molecules (blue).

show abstract

“…Most of the methods used deep learning to precisely localize the blinking single-molecule PSFs of a large number of frames [17][18][19][20][21], which ultimately accelerate the data processing time of SMLM. A comprehensive review of deep learning methods in SMLM can be found in [22]. For multicolor SMLM imaging, Hershko et al [23] and Kim et.…”

Section: Introductionmentioning

confidence: 99%

Accelerating multicolor spectroscopic single-molecule localization microscopy using deep learning

Gaire¹,

Zhang²,

Li³

et al. 2020

Biomed. Opt. Express

View full text Add to dashboard Cite

Spectroscopic single-molecule localization microscopy (sSMLM) simultaneously provides spatial localization and spectral information of individual single-molecules emission, offering multicolor super-resolution imaging of multiple molecules in a single sample with the nanoscopic resolution. However, this technique is limited by the requirements of acquiring a large number of frames to reconstruct a super-resolution image. In addition, multicolor sSMLM imaging suffers from spectral cross-talk while using multiple dyes with relatively broad spectral bands that produce cross-color contamination. Here, we present a computational strategy to accelerate multicolor sSMLM imaging. Our method uses deep convolution neural networks to reconstruct high-density multicolor super-resolution images from low-density, contaminated multicolor images rendered using sSMLM datasets with much fewer frames, without compromising spatial resolution. High-quality, super-resolution images are reconstructed using up to 8-fold fewer frames than usually needed. Thus, our technique generates multicolor super-resolution images within a much shorter time, without any changes in the existing sSMLM hardware system. Two-color and three-color sSMLM experimental results demonstrate superior reconstructions of tubulin/mitochondria, peroxisome/mitochondria, and tubulin/mitochondria/peroxisome in fixed COS-7 and U2-OS cells with a significant reduction in acquisition time.

show abstract

Deep learning in single-molecule microscopy: fundamentals, caveats, and recent developments [Invited]

Cited by 78 publications

References 84 publications

Super-resolution Microscopy with Single Molecules in Biology and Beyond–Essentials, Current Trends, and Future Challenges

Super-resolution Microscopy with Single Molecules in Biology and Beyond–Essentials, Current Trends, and Future Challenges

Plasmonic Assemblies for Real‐Time Single‐Molecule Biosensing

Accelerating multicolor spectroscopic single-molecule localization microscopy using deep learning

Contact Info

Product

Resources

About