Video microscopy has a long history of providing insight and breakthroughs for a broad range of disciplines, from physics to biology. Image analysis to extract quantitative information from video microscopy data has traditionally relied on algorithmic approaches, which are often difficult to implement, time-consuming, and computationally expensive. Recently, alternative data-driven approaches using deep learning have greatly improved quantitative digital microscopy, potentially offering automatized, accurate, and fast image analysis. However, the combination of deep learning and video microscopy remains underutilized primarily due to the steep learning curve involved in developing custom deep-learning solutions. To overcome this issue, we introduce software, DeepTrack 2.0, to design, train, and validate deep-learning solutions for digital microscopy. We use this software to exemplify how deep learning can be employed for a broad range of applications, from particle localization, tracking, and characterization, to cell counting and classification. Thanks to its user-friendly graphical interface, DeepTrack 2.0 can be easily customized for user-specific applications, and thanks to its open-source, object-oriented programing, it can be easily expanded to add features and functionalities, potentially introducing deep-learning-enhanced video microscopy to a far wider audience.

Determination of size and refractive index (RI) of dispersed unlabeled subwavelength particles is of growing interest in several fields, including biotechnology, wastewater monitoring, and nanobubble preparations. Conventionally, the size distribution of such samples is determined via the Brownian motion of the particles, but simultaneous determination of their RI remains challenging. This work demonstrates nanoparticle tracking analysis (NTA) in an off-axis digital holographic microscope (DHM) enabling determination of both particle size and RI of individual subwavelength particles from the combined information about size and optical phase shift. The potential of the method to separate particle populations is demonstrated by analyzing a mixture of three types of dielectric particles within a narrow size range, where conventional NTA methods based on Brownian motion alone would fail. Using this approach, the phase shift allowed individual populations of dielectric beads overlapping in either size or RI to be clearly distinguished and quantified with respect to these properties. The method was furthermore applied for analysis of surfactant-stabilized micro- and nanobubbles, with RI lower than that of water. Since bubbles induce a phase shift of opposite sign to that of solid particles, they were easily distinguished from similarly sized solid particles made up of undissolved surfactant. Surprisingly, the dependence of the phase shift on bubble size indicates that only those with 0.15–0.20 μm radius were individual bubbles, whereas larger bubbles were actually clusters of bubbles. This label-free means to quantify multiple parameters of suspended individual submicrometer particles offers a crucial complement to current characterization strategies, suggesting broad applicability for a wide range of nanoparticle systems.

Characterization of suspended nanoparticles in their native environment plays a central role in a wide range of fields, from medical diagnostics and nanoparticle-enhanced drug delivery to nanosafety and environmental nanopollution assessment. Standard optical approaches for nanoparticle sizing assess the size via the diffusion constant and, as a consequence, require long trajectories and that the medium has a known and uniform viscosity. However, in most biological applications, only short trajectories are available, while simultaneously, the medium viscosity is unknown and tends to display spatiotemporal variations. In this work, we demonstrate a label-free method to quantify not only size but also refractive index of individual subwavelength particles using 2 orders of magnitude shorter trajectories than required by standard methods and without prior knowledge about the physicochemical properties of the medium. We achieved this by developing a weighted average convolutional neural network to analyze holographic images of single particles, which was successfully applied to distinguish and quantify both size and refractive index of subwavelength silica and polystyrene particles without prior knowledge of solute viscosity or refractive index. We further demonstrate how these features make it possible to temporally resolve aggregation dynamics of 31 nm polystyrene nanoparticles, revealing previously unobserved time-resolved dynamics of the monomer number and fractal dimension of individual subwavelength aggregates.

No abstract

The characterization of dynamical processes in living systems provides important clues for their mechanistic interpretation and link to biological functions. Owing to recent advances in microscopy techniques, it is now possible to routinely record the motion of cells, organelles and individual molecules at multiple spatiotemporal scales in physiological conditions. However, the automated analysis of dynamics occurring in crowded and complex environments still lags behind the acquisition of microscopic image sequences. Here we present a framework based on geometric deep learning that achieves the accurate estimation of dynamical properties in various biologically relevant scenarios. This deep-learning approach relies on a graph neural network enhanced by attention-based components. By processing object features with geometric priors, the network is capable of performing multiple tasks, from linking coordinates into trajectories to inferring local and global dynamic properties. We demonstrate the flexibility and reliability of this approach by applying it to real and simulated data corresponding to a broad range of biological experiments.