A deep learning framework for nucleus segmentation using image style transfer

Abstract

Single cell segmentation is typically one of the first and most crucial tasks of image-based cellular analysis. We present a deep learning approach aiming towards a truly general method for localizing nuclei across a diverse range of assays and light microscopy modalities. We outperform the 739 methods submitted to the 2018 Data Science Bowl on images representing a variety of realistic conditions, some of which were not represented in the training data. The key to our approach is to adapt our model to unseen and unlabeled data using image style transfer to generate augmented training samples. This allows the model to recognize nuclei in new and different experiments without requiring expert annotations.Identifying nuclei is the starting point for many microscopy-based cellular analyses. Accurate localization of the nucleus is the basis of a variety of quantitative measurements, but is also a first step for identifying individual cell borders, which enables a multitude of further analyses. Until recently, the dominant approaches for this task have been based on classic image processing algorithms (e.g. CellProfiler 1 ) which were sometimes guided by shape and spatial priors 2 . A drawback of these methods is the need for expert knowledge to properly adjust the parameters, which typically must be re-tuned when experimental conditions change.Recently, deep learning has revolutionized an assortment of tasks in image analysis, from image classification 3 to face recognition 4 , and scene segmentation 5 . It is also responsible for breakthroughs in diagnosing retinal images 6 , classifying skin lesions with superhuman performance 7 , as well as incredible advances in 3D fluorescence image analysis 8 . However, aside from initial works from Caicedo et al. 9 and Van Valen et al. 10 , deep learning has yet to significantly advance nucleus segmentation performance.

Cellpose: a generalist algorithm for cellular segmentation

“…The winning algorithms from the challenge used established com-puter vision algorithms like Mask R-CNN [15,16] and adapted the algorithms for the biological problem. Following the competition, this dataset generated further progress, with other methods like Stardist and nucle-AIzer being developed specifically for this dataset [17,18].…”

Test-time augmentation for deep learning-based cell segmentation on microscopy images

“…Additionally, we also trained U-Net without augmented data to analyse TTA performance on such a network as well. The network, following the strategy described by Hollandi et al 5 , was trained for 3 epochs for different layer groups: first, all network layers were trained at a learning…”

“…Numerous approaches have been developed, such as methods using mathematical morphology 1 or differential geometry 2,3 . More recently deep learning has yielded a never-seen improvement of accuracy and robustness 4,5,6 . Remarkably, Kaggle's Data Science Bowl 2018 (DSB) 7 was dedicated to nuclei segmentation and gave a great momentum to this field.…”

Preliminary Evaluation of the Utility of Deep Generative Histopathology Image Translation at a Mid-Sized NCI Cancer Center

“…Other studies have sought to use GANs to generate synthetic training data to increase the generalizability of deep learning histopathology models . A few studies used deep generative techniques to derive nucleus masks without the use of physician supplied annotations (Bug et al, 2019;Gadermayr et al, 2019;Hollandi et al, 2019;Mahmood et al, 2018).…”