Single-shot self-supervised object detection in microscopy

Abstract

Object detection is a fundamental task in digital microscopy, where machine learning has made great strides in overcoming the limitations of classical approaches. The training of state-of-the-art machine-learning methods almost universally relies on vast amounts of labeled experimental data or the ability to numerically simulate realistic datasets. However, experimental data are often challenging to label and cannot be easily reproduced numerically. Here, we propose a deep-learning method, named LodeSTAR (Localization and detection from Symmetries, Translations And Rotations), that learns to detect microscopic objects with sub-pixel accuracy from a single unlabeled experimental image by exploiting the inherent roto-translational symmetries of this task. We demonstrate that LodeSTAR outperforms traditional methods in terms of accuracy, also when analyzing challenging experimental data containing densely packed cells or noisy backgrounds. Furthermore, by exploiting additional symmetries we show that LodeSTAR can measure other properties, e.g., vertical position and polarizability in holographic microscopy.

Deep-learning enabled single-shot 3D feature detection in fringe projection profilometry

“…In addition, a set of internal transformations, such as translation, rotation, scaling, and smoothing, were applied to increase the dataset during training without requiring additional original images. This process is inspired by the LodeSTAR method, 8 which exploits the similarity between objects to be detected. Because of the three-dimensional nature of the image, more accurate results are obtained with less data than those obtained using conventional approaches.…”

“…The consistency loss is internally calculated to ensure that the predictions are correct and that the results do not depend on the applied transformation or location of the object in the image, 8 and is computed as…”

“…[5][6][7] Recently, a new deep-learning technique that takes advantage of the inherent symmetries in object detection was developed in microscopy. 8 This method allows it to be trained on extremely small datasets, even as little as a single image of the object, without requiring any ground truth. This approach can achieve higher accuracy than traditional methods with only one training image and can also provide reliable detection in challenging experimental settings, such as densely packed or noisy images.…”

“…This approach can achieve higher accuracy than traditional methods with only one training image and can also provide reliable detection in challenging experimental settings, such as densely packed or noisy images. 8 However, its application for multi-scale detection requires further work. This paper proposes a methodology for detecting 3D features of different scales with a single training image.…”

Light, Matter, Action: Shining Light on Active Matter

“…Higher control will enable more fundamental scientific discoveries about far-from-equilibrium phenomena, while being useful for applications, e.g., in sensing, nanomedicine, and materials science . Nowadays, the prospects for light actuation are ever brighter thanks to the development of several new technologies, such as cheaper lasers at all wavelengths, more versatile spatial light modulators, higher-speed cameras, and advanced particle tracking algorithms based on machine learning. − …”